00010000:00000001:3.0F:1782489555.116086:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489555.116106:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489555.116113:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489555.116114:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0F:1782489555.116185:0:238717:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489555.116194:0:238717:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489555.116294:0:238717:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0F:1782489555.371970:0:237024:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304220 (cpu 0) has reached its deadline 4303800000, at 4303800064 00000100:00000010:2.0:1782489555.372001:0:237024:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce587231300. 00000100:00000040:3.0:1782489555.373743:0:213139:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304330 (cpu 0) has reached its deadline 4303800000, at 4303800066 00000100:00000010:3.0:1782489555.373756:0:213139:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce5872314e0. 00010000:00000001:3.0:1782489556.144160:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489556.144170:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489556.144178:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489556.144180:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1782489556.144218:0:249964:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1782489556.144221:0:249964:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1782489556.144224:0:249964:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1782489556.652802:0:174483:0:(pinger.c:231:ptlrpc_pinger_process_import()) dd22dfa7-56fe-4502-9376-64e8b3112148->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1782489556.652814:0:174483:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1782489556.652823:0:174483:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce588dd0000. 00000020:00000040:1.0:1782489556.652829:0:174483:0:(genops.c:1231:class_import_get()) import ffff9ce5834b1800 refcount=4 obd=MGC192.168.202.139@tcp 00000100:00000001:1.0:1782489556.652833:0:174483:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1782489556.652836:0:174483:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1782489556.652841:0:174483:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1782489556.652852:0:174483:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b4e0c700. 00000100:00000001:1.0:1782489556.652860:0:174483:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652874:0:174483:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging dd22dfa7-56fe-4502-9376-64e8b3112148->MGS req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1782489556.652885:0:174483:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1782489556.652889:0:174483:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652895:0:174483:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003d5653ec] to pc [ptlrpcd_00_00+0] req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1782489556.652933:0:174483:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652937:0:174483:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1782489556.652940:0:174483:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1782489556.652943:0:174483:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce588dd3480. 00000020:00000040:1.0:1782489556.652946:0:174483:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3000 refcount=9 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000100:00000001:1.0:1782489556.652949:0:174483:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1782489556.652950:0:174483:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1782489556.652952:0:174483:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1782489556.652957:0:174483:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b4e0c100. 00000100:00000001:1.0:1782489556.652960:0:174483:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652965:0:174483:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1782489556.652972:0:174483:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1782489556.652974:0:174483:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652985:0:174483:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000032a02d41] to pc [ptlrpcd_00_01+1] req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1782489556.652995:0:174483:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.652997:0:174483:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1782489556.653001:0:174483:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1782489556.653003:0:174483:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce588dd2300. 00000020:00000040:1.0:1782489556.653006:0:174483:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc1000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000100:00000001:1.0:1782489556.653008:0:174483:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1782489556.653010:0:174483:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1782489556.653012:0:174483:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1782489556.653015:0:174483:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b4e0c200. 00000100:00000001:1.0:1782489556.653019:0:174483:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653023:0:174483:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1782489556.653030:0:174483:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1782489556.653032:0:174483:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653036:0:174483:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000998f48c] to pc [ptlrpcd_00_00+0] req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1782489556.653056:0:174483:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653058:0:174483:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1782489556.653062:0:174483:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1782489556.653064:0:174483:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce588dd1500. 00000020:00000040:1.0:1782489556.653066:0:174483:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3800 refcount=4 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000100:00000001:1.0:1782489556.653069:0:174483:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1782489556.653070:0:174483:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1782489556.653072:0:174483:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1782489556.653075:0:174483:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b4e0ca00. 00000100:00000001:1.0:1782489556.653077:0:174483:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653081:0:174483:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1782489556.653101:0:174483:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1782489556.653112:0:174483:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653117:0:174483:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000001e10f8a0] to pc [ptlrpcd_00_01+1] req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1782489556.653124:0:174483:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1782489556.653126:0:174483:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1782489556.653130:0:174483:0:(pinger.c:249:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1782489556.653134:0:174483:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (9139) 00000100:00000001:0.0F:1782489556.653285:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.653295:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489556.653297:0:2416:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489556.653304:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653313:0:2416:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489556.653315:0:2416:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653319:0:2416:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489556.653322:0:2416:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489556.653325:0:2416:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489556.653329:0:2416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce588dd0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:dd22dfa7-56fe-4502-9376-64e8b3112148:2416:1869066440683904:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489556.653334:0:2416:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489556.653340:0:2416:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489556.653354:0:2416:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653357:0:2416:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489556.653362:0:2416:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677d600. 02000000:00000001:0.0:1782489556.653364:0:2416:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489556.653371:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489556.653376:0:2416:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a058. 00000400:00000010:0.0:1782489556.653384:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724dbb0. 00000100:00000200:0.0:1782489556.653391:0:2416:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440683904, portal 25 00000100:00000001:0.0:1782489556.653394:0:2416:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489556.653396:0:2416:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108390469632 : -108965319081984 : ffff9ce588dd0000) 00000100:00000040:0.0:1782489556.653401:0:2416:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653409:0:2416:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489556.653412:0:2416:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489556.653415:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d660. 00000100:00000200:0.0:1782489556.653419:0:2416:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1869066440683904, offset 0 00000400:00000200:0.0:1782489556.653426:0:2416:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489556.653470:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.653479:0:2416:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489556.653486:0:2416:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489556.653491:0:2416:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489556.653501:0:2416:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489556.653504:0:2416:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489556.653508:0:2416:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536802:536802:256:4294967295] 192.168.202.139@tcp LPNI seq info [536769:536769:8:4294967295] 00000400:00000200:0.0:1782489556.653513:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.653525:0:2416:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489556.653531:0:2416:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489556.653536:0:2416:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198900. 00000800:00000200:0.0:1782489556.653544:0:2416:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489556.653553:0:2416:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489556.653558:0:2416:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489556.653579:0:2416:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489556.653584:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489556.653587:0:2416:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 2 is 4303805122 00000100:00000010:0.0:1782489556.653592:0:2416:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce587231000. 00000100:00000001:0.0:1782489556.653599:0:2416:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 2 00000100:00000001:0.0:1782489556.653604:0:2416:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 3 is 4303805122 00000100:00000010:0.0:1782489556.653607:0:2416:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce587231de0. 00000100:00000001:0.0:1782489556.653610:0:2416:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 3 00000100:00000001:0.0:1782489556.653613:0:2416:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.653616:0:2416:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.653618:0:2416:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489556.653622:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653630:0:2416:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489556.653631:0:2416:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653633:0:2416:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489556.653635:0:2416:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489556.653636:0:2416:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489556.653648:0:2416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce588dd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684160:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489556.653651:0:2416:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489556.653660:0:2416:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489556.653662:0:2416:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653663:0:2416:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489556.653666:0:2416:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677c400. 02000000:00000001:0.0:1782489556.653668:0:2416:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489556.653672:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489556.653675:0:2416:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a420. 00000400:00000010:0.0:1782489556.653679:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d6e8. 00000100:00000200:0.0:1782489556.653682:0:2416:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684160, portal 10 00000100:00000001:0.0:1782489556.653683:0:2416:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489556.653684:0:2416:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108390478592 : -108965319073024 : ffff9ce588dd2300) 00000100:00000040:0.0:1782489556.653691:0:2416:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653697:0:2416:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489556.653700:0:2416:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489556.653702:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d088. 00000100:00000200:0.0:1782489556.653704:0:2416:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440684160, offset 0 00000400:00000200:0.0:1782489556.653708:0:2416:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489556.653712:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.653715:0:2416:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489556.653718:0:2416:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489556.653721:0:2416:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489556.653723:0:2416:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489556.653725:0:2416:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489556.653727:0:2416:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536803:536803:255:4294967295] 192.168.202.139@tcp LPNI seq info [536770:536770:7:4294967295] 00000400:00000200:0.0:1782489556.653731:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.653736:0:2416:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489556.653740:0:2416:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489556.653743:0:2416:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198000. 00000800:00000200:0.0:1782489556.653746:0:2416:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489556.653750:0:2416:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489556.653752:0:2416:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489556.653761:0:2416:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.653764:0:2416:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: using a timeout of 25115 usecs (26 jiffies) 00000400:00000200:0.0:1782489556.653767:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489556.653768:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303801372 00000100:00000001:0.0:1782489556.653771:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.653772:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303801372 00000100:00000001:0.0:1782489556.653774:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.653776:0:2416:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.653777:0:2416:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.653780:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.653782:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.653789:0:2416:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489556.653791:0:2416:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1782489556.653793:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.653794:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489556.653795:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489556.653798:0:2416:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:0.0:1782489556.653799:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.653801:0:2416:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489556.653802:0:2416:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.653804:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.653804:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489556.653806:0:2416:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000800:00000200:3.0:1782489556.653808:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1782489556.653809:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000010:3.0:1782489556.653813:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198900. 00000100:00000001:0.0:1782489556.653816:0:2416:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489556.653817:0:2416:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489556.653818:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 02000000:00000001:0.0:1782489556.653819:0:2416:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489556.653821:0:2416:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1782489556.653822:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 02000000:00000001:0.0:1782489556.653823:0:2416:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489556.653826:0:2416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce588dd3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684032:192.168.202.139@tcp:400:kworker.0 00000400:00000200:3.0:1782489556.653828:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d660 00000100:00000001:0.0:1782489556.653829:0:2416:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1782489556.653830:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d660. 02000000:00000001:0.0:1782489556.653831:0:2416:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:3.0:1782489556.653833:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1782489556.653837:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1782489556.653837:0:2416:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653839:0:2416:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489556.653841:0:2416:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677ce00. 02000000:00000001:0.0:1782489556.653843:0:2416:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489556.653845:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1782489556.653846:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489556.653848:0:2416:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a948. 00000100:00000040:3.0:1782489556.653849:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1782489556.653852:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d880. 00000100:00000200:0.0:1782489556.653854:0:2416:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684032, portal 10 00000100:00000001:3.0:1782489556.653856:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.653856:0:2416:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489556.653858:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1782489556.653858:0:2416:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108390483072 : -108965319068544 : ffff9ce588dd3480) 00000100:00000040:0.0:1782489556.653863:0:2416:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653869:0:2416:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:3.0:1782489556.653870:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000040:0.0:1782489556.653871:0:2416:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489556.653872:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198000. 00000400:00000010:0.0:1782489556.653873:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724dee0. 00000400:00000200:3.0:1782489556.653874:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000200:0.0:1782489556.653875:0:2416:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440684032, offset 0 00000400:00000200:3.0:1782489556.653877:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:0.0:1782489556.653879:0:2416:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489556.653881:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d088 00000400:00000010:3.0:1782489556.653883:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d088. 00000400:00000200:0.0:1782489556.653883:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489556.653885:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000400:00000200:0.0:1782489556.653886:0:2416:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000100:00000200:3.0:1782489556.653888:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1782489556.653889:0:2416:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489556.653891:0:2416:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489556.653894:0:2416:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489556.653896:0:2416:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489556.653899:0:2416:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536804:536804:255:4294967295] 192.168.202.139@tcp LPNI seq info [536771:536771:7:4294967295] 00000100:00000001:3.0:1782489556.653901:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000400:00000200:0.0:1782489556.653903:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000040:3.0:1782489556.653904:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1782489556.653909:0:2416:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000100:00000001:3.0:1782489556.653911:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489556.653912:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1782489556.653914:0:2416:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489556.653917:0:2416:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198100. 00000800:00000200:0.0:1782489556.653920:0:2416:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489556.653924:0:2416:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489556.653926:0:2416:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489556.653938:0:2416:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.653941:0:2416:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: using a timeout of 25128 usecs (26 jiffies) 00000400:00000200:0.0:1782489556.653944:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489556.653946:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303801372 00000100:00000001:0.0:1782489556.653948:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.653951:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303801372 00000100:00000001:0.0:1782489556.653952:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.653954:0:2416:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.653957:0:2416:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.653959:0:2416:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489556.653963:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:3.0:1782489556.653964:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1782489556.653967:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198100. 00000400:00000200:3.0:1782489556.653970:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1782489556.653970:0:2416:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489556.653971:0:2416:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489556.653973:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 02000000:00000001:0.0:1782489556.653973:0:2416:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489556.653975:0:2416:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:3.0:1782489556.653976:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724dee0 02000000:00000001:0.0:1782489556.653977:0:2416:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:3.0:1782489556.653978:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724dee0. 00000100:00000001:3.0:1782489556.653980:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00100000:0.0:1782489556.653980:0:2416:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce588dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684288:192.168.202.139@tcp:400:kworker.0 00000100:00000200:3.0:1782489556.653983:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.653983:0:2416:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489556.653985:0:2416:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489556.653986:0:2416:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.653988:0:2416:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000001:3.0:1782489556.653990:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 02000000:00000010:0.0:1782489556.653991:0:2416:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677dc00. 00000100:00000040:3.0:1782489556.653992:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1782489556.653993:0:2416:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489556.653996:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489556.653998:0:2416:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687ab58. 00000100:00000001:3.0:1782489556.653999:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489556.654000:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000010:0.0:1782489556.654002:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724df68. 00000100:00000200:0.0:1782489556.654004:0:2416:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684288, portal 4 00000100:00000001:0.0:1782489556.654006:0:2416:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489556.654008:0:2416:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108390475008 : -108965319076608 : ffff9ce588dd1500) 00000100:00000040:0.0:1782489556.654012:0:2416:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.654024:0:2416:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489556.654026:0:2416:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489556.654029:0:2416:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724dee0. 00000100:00000200:0.0:1782489556.654031:0:2416:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1869066440684288, offset 0 00000400:00000200:0.0:1782489556.654035:0:2416:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489556.654039:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.654042:0:2416:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489556.654045:0:2416:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489556.654047:0:2416:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489556.654049:0:2416:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489556.654051:0:2416:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489556.654054:0:2416:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536805:536805:256:4294967295] 192.168.202.139@tcp LPNI seq info [536772:536772:8:4294967295] 00000400:00000200:0.0:1782489556.654057:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.654063:0:2416:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489556.654067:0:2416:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489556.654070:0:2416:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198500. 00000800:00000200:0.0:1782489556.654074:0:2416:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489556.654078:0:2416:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489556.654081:0:2416:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489556.654096:0:2416:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.654100:0:2416:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: using a timeout of 92132 usecs (93 jiffies) 00000400:00000200:0.0:1782489556.654117:0:2416:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489556.654120:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303801439 00000100:00000001:0.0:1782489556.654122:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.654123:0:2416:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303801439 00000100:00000001:0.0:1782489556.654125:0:2416:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303805122 00000100:00000001:0.0:1782489556.654127:0:2416:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.654129:0:2416:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.654137:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.654139:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.654150:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.654152:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1782489556.654334:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1782489556.654337:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198500. 00000400:00000200:3.0:1782489556.654341:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489556.654345:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:3.0:1782489556.654348:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724dee0 00000400:00000010:3.0:1782489556.654350:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724dee0. 00000100:00000001:3.0:1782489556.654353:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1782489556.654356:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489556.654364:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489556.654367:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489556.654386:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489556.654387:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1782489556.664053:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.664073:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489556.664078:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.664080:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.664093:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.664102:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489556.664112:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.664117:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.664124:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.664129:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423280 00000400:00000200:3.0:1782489556.664134:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.664141:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c1d [1] + 224 00000800:00000001:3.0:1782489556.664149:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.664156:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.664159:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489556.664164:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489556.664172:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489556.664180:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489556.664191:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489556.664208:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489556.664219:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.664223:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.664839:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.664846:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1782489556.664856:0:2416:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489556.664858:0:2416:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489556.664861:0:2416:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a420. 00000400:00000200:0.0:1782489556.664866:0:2416:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d6e8 00000400:00000010:0.0:1782489556.664869:0:2416:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d6e8. 00000100:00000001:0.0:1782489556.664873:0:2416:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489556.664879:0:2416:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489556.664890:0:2416:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.664896:0:2416:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489556.664898:0:2416:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.664901:0:2416:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489556.664905:0:2416:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489556.664907:0:2416:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489556.664909:0:2416:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.664913:0:2416:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489556.664924:0:2416:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc1460 time=37 v=60 (60 60 60 60) 00000100:00001000:0.0:1782489556.664928:0:2416:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc1418 time=61 v=5 (1 1 1 1) 00000100:00000001:0.0:1782489556.664930:0:2416:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489556.664932:0:2416:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489556.664934:0:2416:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1782489556.664939:0:2416:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1782489556.664947:0:2416:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.664951:0:2416:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489556.664953:0:2416:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ce5830d8800: skip recheck: last_committed 21474836540 00000100:00000001:0.0:1782489556.664956:0:2416:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489556.664957:0:2416:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.664969:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.664975:0:2416:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.664979:0:2416:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489556.664981:0:2416:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489556.664984:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489556.664992:0:2416:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce588dd2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684160:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489556.664997:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.665000:0:2416:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489556.665003:0:2416:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce588dd2300 x1869066440684160/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.665011:0:2416:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489556.665014:0:2416:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489556.665017:0:2416:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677c400. 02000000:00000001:0.0:1782489556.665023:0:2416:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489556.665025:0:2416:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489556.665027:0:2416:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc1000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000020:00000001:0.0:1782489556.665030:0:2416:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489556.665032:0:2416:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b4e0c200. 02000000:00000001:0.0:1782489556.665037:0:2416:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489556.665038:0:2416:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489556.665056:0:2416:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce588dd2300. 00000100:00000001:0.0:1782489556.665060:0:2416:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489556.665062:0:2416:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.665066:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.665067:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.665068:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489556.665070:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.665071:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1782489556.669873:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.669891:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489556.669896:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.669898:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.669908:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.669930:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489556.669940:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.669944:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.669949:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.669954:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423200 00000400:00000200:3.0:1782489556.669958:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.669964:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c25 [1] + 224 00000800:00000001:3.0:1782489556.669970:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.669975:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.669979:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489556.669983:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489556.669990:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489556.669998:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489556.670010:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489556.670032:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489556.670041:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.670045:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489556.670047:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.670049:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.670053:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670056:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489556.670061:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670063:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670066:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670068:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 25 MB=0x6a3e7f1423180 00000400:00000200:3.0:1782489556.670072:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670076:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c15 [1] + 224 00000800:00000001:3.0:1782489556.670080:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.670083:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.670090:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489556.670093:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489556.670096:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489556.670101:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489556.670110:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489556.670117:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489556.670121:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.670125:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670125:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:3.0:1782489556.670127:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489556.670129:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670130:0:2416:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000200:3.0:1782489556.670131:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670134:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000001:0.0:1782489556.670136:0:2416:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:3.0:1782489556.670139:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670141:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000001:0.0:1782489556.670143:0:2416:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:3.0:1782489556.670144:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489556.670146:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 4 MB=0x6a3e7f1423300 00000400:00000010:0.0:1782489556.670146:0:2416:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a948. 00000400:00000200:3.0:1782489556.670150:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489556.670151:0:2416:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d880 00000400:00000010:0.0:1782489556.670153:0:2416:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d880. 00000100:00000001:0.0:1782489556.670156:0:2416:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489556.670160:0:2416:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1782489556.670161:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c2d [1] + 224 00000800:00000001:3.0:1782489556.670165:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1782489556.670169:0:2416:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:3.0:1782489556.670171:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489556.670173:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489556.670176:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:0.0:1782489556.670176:0:2416:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489556.670178:0:2416:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489556.670179:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1782489556.670181:0:2416:0:(client.c:1579:after_reply()) Process entered 00000100:00000200:3.0:1782489556.670183:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1782489556.670185:0:2416:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489556.670187:0:2416:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489556.670190:0:2416:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489556.670191:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1782489556.670193:0:2416:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489556.670198:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489556.670201:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00001000:0.0:1782489556.670203:0:2416:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc3460 time=37 v=60 (60 60 60 60) 00000100:00001000:0.0:1782489556.670207:0:2416:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3418 time=73 v=5 (1 1 1 1) 00000800:00000001:3.0:1782489556.670208:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670210:0:2416:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489556.670211:0:2416:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489556.670214:0:2416:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489556.670218:0:2416:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.670222:0:2416:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489556.670224:0:2416:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ce5830d8800: skip recheck: last_committed 30064771619 00000100:00000001:0.0:1782489556.670226:0:2416:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489556.670227:0:2416:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.670231:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670237:0:2416:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.670240:0:2416:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489556.670241:0:2416:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489556.670245:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489556.670274:0:2416:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce588dd3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684032:192.168.202.139@tcp:400:kworker.0 00000400:00000001:0.0:1782489556.670280:0:2416:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489556.670281:0:2416:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489556.670283:0:2416:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687ab58. 00000400:00000200:0.0:1782489556.670285:0:2416:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724df68 00000400:00000010:0.0:1782489556.670287:0:2416:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724df68. 00000100:00000001:0.0:1782489556.670289:0:2416:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489556.670292:0:2416:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489556.670301:0:2416:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670308:0:2416:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489556.670310:0:2416:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670311:0:2416:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489556.670313:0:2416:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489556.670315:0:2416:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489556.670316:0:2416:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.670318:0:2416:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489556.670323:0:2416:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5b6bc3c60 time=37 v=5 (5 5 5 5) 00000100:00001000:0.0:1782489556.670326:0:2416:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3c18 time=114 v=5 (1 1 0 0) 00000100:00000001:0.0:1782489556.670329:0:2416:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489556.670330:0:2416:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489556.670332:0:2416:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489556.670334:0:2416:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.670336:0:2416:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489556.670338:0:2416:0:(client.c:3107:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ce5830d8800: skip recheck: last_committed 85899368364 00000100:00000001:0.0:1782489556.670340:0:2416:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489556.670341:0:2416:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.670356:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670363:0:2416:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.670371:0:2416:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489556.670372:0:2416:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489556.670377:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489556.670385:0:2416:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce588dd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2416:1869066440684288:192.168.202.139@tcp:400:kworker.0 00000400:00000001:0.0:1782489556.670390:0:2416:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489556.670392:0:2416:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489556.670393:0:2416:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a058. 00000400:00000200:0.0:1782489556.670396:0:2416:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724dbb0 00000400:00000010:0.0:1782489556.670398:0:2416:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724dbb0. 00000100:00000001:0.0:1782489556.670400:0:2416:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489556.670403:0:2416:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489556.670410:0:2416:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670416:0:2416:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489556.670417:0:2416:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670418:0:2416:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489556.670420:0:2416:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489556.670421:0:2416:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489556.670422:0:2416:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489556.670424:0:2416:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489556.670426:0:2416:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5834b1c60 time=37 v=5 (5 5 5 5) 00000100:00001000:0.0:1782489556.670757:0:2416:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5834b1c18 time=73 v=5 (1 1 1 1) 00000100:00000001:0.0:1782489556.670760:0:2416:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489556.670762:0:2416:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489556.670763:0:2416:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489556.670765:0:2416:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.670767:0:2416:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489556.670771:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670778:0:2416:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489556.670785:0:2416:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489556.670787:0:2416:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489556.670791:0:2416:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489556.670798:0:2416:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce588dd0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:dd22dfa7-56fe-4502-9376-64e8b3112148:2416:1869066440683904:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489556.670801:0:2416:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670804:0:2416:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489556.670807:0:2416:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce588dd3480 x1869066440684032/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489641 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670813:0:2416:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489556.670816:0:2416:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489556.670820:0:2416:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677ce00. 02000000:00000001:0.0:1782489556.670824:0:2416:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489556.670827:0:2416:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489556.670829:0:2416:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3000 refcount=8 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000020:00000001:0.0:1782489556.670831:0:2416:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489556.670833:0:2416:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b4e0c100. 02000000:00000001:0.0:1782489556.670837:0:2416:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489556.670839:0:2416:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489556.670841:0:2416:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce588dd3480. 00000100:00000001:0.0:1782489556.670844:0:2416:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489556.670846:0:2416:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670848:0:2416:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489556.670850:0:2416:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce588dd1500 x1869066440684288/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670856:0:2416:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489556.670858:0:2416:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489556.670859:0:2416:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677dc00. 02000000:00000001:0.0:1782489556.670862:0:2416:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489556.670863:0:2416:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489556.670865:0:2416:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3800 refcount=3 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000020:00000001:0.0:1782489556.670867:0:2416:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489556.670868:0:2416:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b4e0ca00. 02000000:00000001:0.0:1782489556.670875:0:2416:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489556.670876:0:2416:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489556.670878:0:2416:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce588dd1500. 00000100:00000001:0.0:1782489556.670880:0:2416:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489556.670881:0:2416:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670883:0:2416:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489556.670886:0:2416:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce588dd0000 x1869066440683904/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489572 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489556.670891:0:2416:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489556.670893:0:2416:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489556.670894:0:2416:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677d600. 02000000:00000001:0.0:1782489556.670896:0:2416:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489556.670897:0:2416:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489556.670898:0:2416:0:(genops.c:1244:class_import_put()) import ffff9ce5834b1800 refcount=3 obd=MGC192.168.202.139@tcp 00000020:00000001:0.0:1782489556.670899:0:2416:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489556.670901:0:2416:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b4e0c700. 02000000:00000001:0.0:1782489556.670902:0:2416:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489556.670903:0:2416:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489556.670904:0:2416:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce588dd0000. 00000100:00000001:0.0:1782489556.670906:0:2416:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489556.670907:0:2416:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670908:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489556.670916:0:2416:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489556.670917:0:2416:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489556.670920:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489556.670922:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489557.164165:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489557.164170:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489557.164176:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489557.164177:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489557.164248:0:238718:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489557.164251:0:238718:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489557.164255:0:238718:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489558.187893:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489558.187903:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489558.187910:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489558.187911:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1782489558.187935:0:254078:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1782489558.187938:0:254078:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1782489558.187941:0:254078:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489559.212542:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489559.212550:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489559.212556:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489559.212558:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1782489559.213471:0:244254:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1782489559.213475:0:244254:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1782489559.213480:0:244254:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489560.235634:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489560.235643:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489560.235651:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489560.235656:0:213139:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:3.0:1782489560.235659:0:213139:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:3.0:1782489560.235667:0:213139:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1782489560.235670:0:213139:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1782489560.235672:0:213139:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489560.235676:0:213139:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:3.0:1782489560.235679:0:213139:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489560.235681:0:213139:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489560.235683:0:213139:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1782489560.236681:0:2407:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce5c216a6c0] -> 12345-192.168.202.39@tcp (6) 00000800:00000200:2.0:1782489560.236690:0:2407:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.39@tcp ip 192.168.202.39:988 00000800:00000200:2.0:1782489560.236696:0:2407:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce58a47e800 type 192, nob 24 niov 1 nkiov 0 00000020:00000001:1.0:1782489560.236882:0:238717:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489560.236890:0:238717:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489560.236895:0:238717:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000200:3.0:1782489560.237253:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000001:3.0:1782489560.237261:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489560.237277:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489560.237281:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489560.237284:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489560.491611:0:213139:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304330 (cpu 0) has reached its deadline 4303805122, at 4303805184 00000100:00000010:3.0:1782489560.491633:0:213139:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce587231de0. 00000100:00000040:2.0:1782489560.493121:0:237024:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304220 (cpu 0) has reached its deadline 4303805122, at 4303805185 00000100:00000010:2.0:1782489560.493137:0:237024:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce587231000. 00010000:00000001:3.0:1782489561.259698:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489561.259706:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489561.259715:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489561.259739:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1782489561.259824:0:249964:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1782489561.259827:0:249964:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1782489561.259830:0:249964:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1782489561.775529:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) dd22dfa7-56fe-4502-9376-64e8b3112148->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1782489561.775545:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1782489561.775561:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5a621d180. 00000020:00000040:0.0:1782489561.775571:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5834b1800 refcount=4 obd=MGC192.168.202.139@tcp 00000100:00000001:0.0:1782489561.775576:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1782489561.775580:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1782489561.775587:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1782489561.775630:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5ba198500. 00000100:00000001:0.0:1782489561.775640:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775657:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging dd22dfa7-56fe-4502-9376-64e8b3112148->MGS req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1782489561.775670:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1782489561.775676:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775684:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000009086ea9f] to pc [ptlrpcd_00_00+0] req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.775727:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775730:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1782489561.775734:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1782489561.775737:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5a621c700. 00000020:00000040:0.0:1782489561.775741:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3000 refcount=9 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000100:00000001:0.0:1782489561.775743:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1782489561.775744:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1782489561.775746:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1782489561.775750:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5ba198700. 00000100:00000001:0.0:1782489561.775753:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775757:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1782489561.775765:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1782489561.775778:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775789:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000394be1f8] to pc [ptlrpcd_00_01+1] req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.775799:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775801:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1782489561.775804:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1782489561.775807:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5a621c000. 00000020:00000040:0.0:1782489561.775809:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc1000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000100:00000001:0.0:1782489561.775812:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1782489561.775819:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1782489561.775820:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1782489561.775825:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5ba198100. 00000100:00000001:0.0:1782489561.775828:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775833:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1782489561.775840:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1782489561.775842:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775846:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000407613dc] to pc [ptlrpcd_00_00+0] req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.775853:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.775855:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1782489561.775858:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1782489561.775861:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5a621c380. 00000020:00000040:0.0:1782489561.776163:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3800 refcount=4 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000100:00000001:0.0:1782489561.776166:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1782489561.776167:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1782489561.776169:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1782489561.776173:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5ba198000. 00000100:00000001:0.0:1782489561.776177:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.776182:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1782489561.776190:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1782489561.776192:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.776197:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ddbf36a1] to pc [ptlrpcd_00_01+1] req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776204:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.776206:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1782489561.776210:0:284611:0:(pinger.c:249:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1782489561.776233:0:284611:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (9144) 00000100:00000001:0.0:1782489561.776255:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.776259:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489561.776262:0:2417:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489561.776269:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776277:0:2417:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489561.776280:0:2417:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.776285:0:2417:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489561.776288:0:2417:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489561.776291:0:2417:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489561.776295:0:2417:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5a621c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684544:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.776299:0:2417:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489561.776306:0:2417:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489561.776310:0:2417:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.776314:0:2417:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489561.776319:0:2417:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677d200. 02000000:00000001:0.0:1782489561.776321:0:2417:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489561.776330:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489561.776334:0:2417:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a058. 00000400:00000010:0.0:1782489561.776343:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724dbb0. 00000100:00000200:0.0:1782489561.776352:0:2417:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684544, portal 10 00000100:00000001:0.0:1782489561.776355:0:2417:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489561.776356:0:2417:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108881516288 : -108964828035328 : ffff9ce5a621c700) 00000100:00000040:0.0:1782489561.776365:0:2417:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776374:0:2417:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489561.776376:0:2417:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489561.776379:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724df68. 00000100:00000200:0.0:1782489561.776382:0:2417:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440684544, offset 0 00000400:00000200:0.0:1782489561.776389:0:2417:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489561.776400:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.776409:0:2417:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489561.776421:0:2417:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489561.776426:0:2417:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489561.776476:0:2417:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489561.776479:0:2417:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489561.776483:0:2417:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536806:536806:256:4294967295] 192.168.202.139@tcp LPNI seq info [536773:536773:8:4294967295] 00000400:00000200:0.0:1782489561.776487:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.776500:0:2417:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489561.776507:0:2417:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489561.776513:0:2417:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198900. 00000800:00000200:0.0:1782489561.776521:0:2417:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489561.776530:0:2417:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489561.776536:0:2417:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489561.776558:0:2417:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.776565:0:2417:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: using a timeout of 25127 usecs (26 jiffies) 00000400:00000200:0.0:1782489561.776569:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489561.776572:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303806495 00000100:00000010:0.0:1782489561.776577:0:2417:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce587231960. 00000100:00000001:0.0:1782489561.776584:0:2417:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request now active for cpu 2 00000100:00000001:0.0:1782489561.776594:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303806495 00000100:00000010:0.0:1782489561.776598:0:2417:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce587231ae0. 00000100:00000001:0.0:1782489561.776601:0:2417:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request now active for cpu 3 00000100:00000001:0.0:1782489561.776612:0:2417:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.776616:0:2417:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.776619:0:2417:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489561.776623:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776630:0:2417:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489561.776631:0:2417:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.776633:0:2417:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489561.776641:0:2417:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489561.776643:0:2417:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489561.776647:0:2417:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5a621c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684800:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.776651:0:2417:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489561.776654:0:2417:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489561.776655:0:2417:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.776657:0:2417:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489561.776660:0:2417:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677d400. 02000000:00000001:0.0:1782489561.776663:0:2417:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489561.776667:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489561.776671:0:2417:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687ab58. 00000400:00000010:0.0:1782489561.776677:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d880. 00000100:00000200:0.0:1782489561.776680:0:2417:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684800, portal 4 00000100:00000001:0.0:1782489561.776682:0:2417:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489561.776684:0:2417:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108881515392 : -108964828036224 : ffff9ce5a621c380) 00000100:00000040:0.0:1782489561.776694:0:2417:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776701:0:2417:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489561.776703:0:2417:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489561.776706:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d6e8. 00000100:00000200:0.0:1782489561.776708:0:2417:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1869066440684800, offset 0 00000400:00000200:0.0:1782489561.776713:0:2417:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489561.776717:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.776721:0:2417:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489561.776723:0:2417:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489561.776726:0:2417:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489561.776728:0:2417:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489561.776730:0:2417:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489561.776733:0:2417:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536807:536807:255:4294967295] 192.168.202.139@tcp LPNI seq info [536774:536774:7:4294967295] 00000400:00000200:0.0:1782489561.776736:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.776743:0:2417:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489561.776748:0:2417:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489561.776757:0:2417:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198f00. 00000800:00000200:0.0:1782489561.776761:0:2417:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489561.776766:0:2417:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489561.776768:0:2417:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489561.776772:0:2417:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.776788:0:2417:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: using a timeout of 92068 usecs (93 jiffies) 00000400:00000200:0.0:1782489561.776792:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489561.776794:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303806562 00000100:00000001:0.0:1782489561.776800:0:2417:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 2, simply extend its deadline from 4303806495 00000100:00000001:0.0:1782489561.776803:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303806562 00000100:00000001:0.0:1782489561.776805:0:2417:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 3, simply extend its deadline from 4303806495 00000100:00000001:0.0:1782489561.776808:0:2417:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.776810:0:2417:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.776813:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.776816:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.776824:0:2417:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489561.776827:0:2417:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1782489561.776829:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.776831:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489561.776833:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489561.776836:0:2417:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:0.0:1782489561.776837:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.776840:0:2417:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489561.776841:0:2417:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.776842:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.776844:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489561.776845:0:2417:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489561.776849:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.776855:0:2417:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489561.776856:0:2417:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.776858:0:2417:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489561.776860:0:2417:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489561.778510:0:2417:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489561.778523:0:2417:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5a621d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2417:1869066440684416:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.778529:0:2417:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489561.778537:0:2417:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489561.778541:0:2417:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.778543:0:2417:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489561.778551:0:2417:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677c600. 02000000:00000001:0.0:1782489561.778554:0:2417:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489561.778561:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489561.778566:0:2417:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a948. 00000400:00000010:0.0:1782489561.778574:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d088. 00000100:00000200:0.0:1782489561.778581:0:2417:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684416, portal 25 00000100:00000001:0.0:1782489561.778583:0:2417:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489561.778584:0:2417:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108881518976 : -108964828032640 : ffff9ce5a621d180) 00000100:00000040:0.0:1782489561.778591:0:2417:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.778600:0:2417:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489561.778603:0:2417:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489561.778606:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d660. 00000100:00000200:0.0:1782489561.778610:0:2417:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1869066440684416, offset 0 00000400:00000200:0.0:1782489561.778618:0:2417:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489561.778625:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.778634:0:2417:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489561.778640:0:2417:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489561.778644:0:2417:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489561.778649:0:2417:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489561.778652:0:2417:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489561.778655:0:2417:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536808:536808:254:4294967295] 192.168.202.139@tcp LPNI seq info [536775:536775:6:4294967295] 00000400:00000200:0.0:1782489561.778659:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.778669:0:2417:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489561.778675:0:2417:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489561.778694:0:2417:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198400. 00000800:00000200:0.0:1782489561.778702:0:2417:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489561.778711:0:2417:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489561.778716:0:2417:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489561.778721:0:2417:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489561.778726:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489561.778729:0:2417:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 2 is 4303810247 00000100:00000001:0.0:1782489561.778738:0:2417:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request active for cpu 2, simply extend its deadline from 4303806562 00000100:00000001:0.0:1782489561.778745:0:2417:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 3 is 4303810247 00000100:00000001:0.0:1782489561.778749:0:2417:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request active for cpu 3, simply extend its deadline from 4303806562 00000100:00000001:0.0:1782489561.778751:0:2417:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.778754:0:2417:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.778758:0:2417:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1782489561.778764:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1782489561.778895:0:2416:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1782489561.778902:0:2416:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1782489561.779693:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489561.779701:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198900. 00000400:00000200:2.0:1782489561.779711:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.779719:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489561.779726:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724df68 00000400:00000010:2.0:1782489561.779729:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724df68. 00000100:00000001:2.0:1782489561.779735:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489561.779743:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.779756:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489561.779759:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.779776:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489561.779779:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489561.779794:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489561.779797:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198f00. 00000400:00000200:2.0:1782489561.779799:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.779803:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489561.779806:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d6e8 00000400:00000010:2.0:1782489561.779807:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d6e8. 00000100:00000001:2.0:1782489561.779810:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489561.779814:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.779821:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489561.779823:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.779844:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489561.779845:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489561.779936:0:2411:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489561.779939:0:2411:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198400. 00000400:00000200:2.0:1782489561.779942:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.780162:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489561.780166:0:2411:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d660 00000400:00000010:2.0:1782489561.780168:0:2411:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d660. 00000100:00000001:2.0:1782489561.780171:0:2411:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489561.780174:0:2411:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.780180:0:2411:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489561.780182:0:2411:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.780186:0:2411:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489561.780187:0:2411:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1782489561.780308:0:2417:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1782489561.780312:0:2417:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.780318:0:2417:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1782489561.780322:0:2417:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1782489561.780326:0:2417:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1782489561.780331:0:2417:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5a621c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684672:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.780335:0:2417:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1782489561.780342:0:2417:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1782489561.780345:0:2417:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.780348:0:2417:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1782489561.780355:0:2417:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce58677c000. 02000000:00000001:0.0:1782489561.780358:0:2417:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1782489561.780363:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:0.0:1782489561.780368:0:2417:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5b687a420. 00000400:00000010:0.0:1782489561.780374:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724da18. 00000100:00000200:0.0:1782489561.780382:0:2417:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684672, portal 10 00000100:00000001:0.0:1782489561.780391:0:2417:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1782489561.780393:0:2417:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635108881514496 : -108964828037120 : ffff9ce5a621c000) 00000100:00000040:0.0:1782489561.780402:0:2417:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.780410:0:2417:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1782489561.780412:0:2417:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:0.0:1782489561.780415:0:2417:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5b724d4c8. 00000100:00000200:0.0:1782489561.780419:0:2417:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440684672, offset 0 00000400:00000200:0.0:1782489561.780426:0:2417:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:0.0:1782489561.780452:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.780459:0:2417:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:0.0:1782489561.780464:0:2417:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1782489561.780468:0:2417:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:0.0:1782489561.780472:0:2417:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1782489561.780475:0:2417:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:0.0:1782489561.780479:0:2417:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536809:536809:256:4294967295] 192.168.202.139@tcp LPNI seq info [536776:536776:8:4294967295] 00000400:00000200:0.0:1782489561.780484:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:0.0:1782489561.780492:0:2417:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:0.0:1782489561.780498:0:2417:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:0.0:1782489561.780503:0:2417:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5ba198b00. 00000800:00000200:0.0:1782489561.780765:0:2417:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:0.0:1782489561.780775:0:2417:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:0.0:1782489561.780779:0:2417:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5ba198b00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1782489561.780881:0:2417:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.780887:0:2417:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: using a timeout of 25113 usecs (26 jiffies) 00000400:00000200:0.0:1782489561.780890:0:2417:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:0.0:1782489561.780893:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303806499 00000100:00000001:0.0:1782489561.780897:0:2417:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303810247 00000100:00000001:0.0:1782489561.780900:0:2417:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303806499 00000100:00000001:0.0:1782489561.780902:0:2417:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303810247 00000100:00000001:0.0:1782489561.780912:0:2417:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.780914:0:2417:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.780919:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.780924:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1782489561.782492:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1782489561.782640:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5ba198b00. 00000400:00000200:3.0:1782489561.782652:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489561.782663:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:3.0:1782489561.782672:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d4c8 00000400:00000010:3.0:1782489561.782675:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d4c8. 00000100:00000001:3.0:1782489561.782681:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1782489561.782690:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489561.782704:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489561.782708:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489561.782715:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489561.782717:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1782489561.789141:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789170:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489561.789177:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789180:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.789194:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789204:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489561.789215:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789235:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789242:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789247:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 25 MB=0x6a3e7f1423380 00000400:00000200:2.0:1782489561.789253:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789261:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c45 [1] + 224 00000800:00000001:2.0:1782489561.789271:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789277:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.789281:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.789286:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489561.789317:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489561.789330:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489561.789342:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.789423:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.789468:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1782489561.789523:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489561.789536:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789540:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489561.789543:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789545:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.789548:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789552:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489561.789557:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789560:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789563:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789566:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 4 MB=0x6a3e7f1423500 00000400:00000200:2.0:1782489561.789569:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.789574:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c3d [1] + 224 00000800:00000001:2.0:1782489561.789578:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789586:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.789587:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.789590:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489561.789594:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489561.789600:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489561.789609:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.789616:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489561.789620:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.789624:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1782489561.789707:0:2417:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489561.789710:0:2417:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489561.789715:0:2417:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a948. 00000400:00000200:0.0:1782489561.789721:0:2417:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d088 00000400:00000010:0.0:1782489561.789723:0:2417:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d088. 00000100:00000001:0.0:1782489561.789728:0:2417:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489561.789733:0:2417:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489561.789744:0:2417:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.789750:0:2417:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489561.789752:0:2417:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.789755:0:2417:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489561.789760:0:2417:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489561.789763:0:2417:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489561.789766:0:2417:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.789846:0:2417:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489561.789855:0:2417:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5834b1c60 time=42 v=5 (5 5 5 5) 00000100:00001000:0.0:1782489561.789860:0:2417:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5834b1c18 time=78 v=5 (1 1 1 1) 00000100:00000001:0.0:1782489561.790308:0:2417:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489561.790310:0:2417:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489561.790314:0:2417:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489561.790317:0:2417:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.790320:0:2417:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.790329:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.790339:0:2417:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.790343:0:2417:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489561.790364:0:2417:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489561.790369:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489561.790377:0:2417:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5a621d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2417:1869066440684416:192.168.202.139@tcp:400:kworker.0 00000400:00000001:0.0:1782489561.790388:0:2417:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489561.790390:0:2417:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489561.790391:0:2417:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687ab58. 00000400:00000200:0.0:1782489561.790396:0:2417:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724d880 00000400:00000010:0.0:1782489561.790398:0:2417:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724d880. 00000100:00000001:0.0:1782489561.790401:0:2417:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489561.790405:0:2417:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489561.790413:0:2417:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.790419:0:2417:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489561.790421:0:2417:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.790423:0:2417:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489561.790424:0:2417:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489561.790426:0:2417:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489561.790427:0:2417:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.790464:0:2417:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489561.790473:0:2417:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5b6bc3c60 time=42 v=5 (5 5 5 5) 00000100:00001000:0.0:1782489561.790477:0:2417:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3c18 time=119 v=5 (1 1 0 0) 00000100:00000001:0.0:1782489561.790479:0:2417:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489561.790482:0:2417:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489561.790484:0:2417:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489561.790488:0:2417:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.790492:0:2417:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489561.790495:0:2417:0:(client.c:3107:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ce5830d8800: skip recheck: last_committed 85899368364 00000100:00000001:0.0:1782489561.790496:0:2417:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489561.790498:0:2417:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.790502:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.790520:0:2417:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.790522:0:2417:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489561.790524:0:2417:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489561.790528:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489561.790535:0:2417:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5a621c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684800:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.790540:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.790543:0:2417:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489561.790547:0:2417:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5a621d180 x1869066440684416/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.790555:0:2417:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489561.790558:0:2417:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489561.790563:0:2417:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677c600. 02000000:00000001:0.0:1782489561.790569:0:2417:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489561.790573:0:2417:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489561.790575:0:2417:0:(genops.c:1244:class_import_put()) import ffff9ce5834b1800 refcount=3 obd=MGC192.168.202.139@tcp 00000020:00000001:0.0:1782489561.790577:0:2417:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489561.790580:0:2417:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5ba198500. 02000000:00000001:0.0:1782489561.790585:0:2417:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489561.790587:0:2417:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489561.790589:0:2417:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5a621d180. 00000100:00000001:0.0:1782489561.790594:0:2417:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489561.790595:0:2417:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.790597:0:2417:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489561.790600:0:2417:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5a621c380 x1869066440684800/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489577 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.790607:0:2417:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489561.790608:0:2417:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489561.790610:0:2417:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677d400. 02000000:00000001:0.0:1782489561.790614:0:2417:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489561.790615:0:2417:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489561.790622:0:2417:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3800 refcount=3 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000020:00000001:0.0:1782489561.790624:0:2417:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489561.790626:0:2417:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5ba198000. 02000000:00000001:0.0:1782489561.790628:0:2417:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489561.790629:0:2417:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489561.790631:0:2417:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5a621c380. 00000100:00000001:0.0:1782489561.790633:0:2417:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489561.790635:0:2417:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.790641:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.790643:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.790646:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1782489561.790648:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.790650:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1782489561.794397:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489561.794423:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489561.794478:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489561.794483:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489561.794494:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489561.794504:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489561.794514:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489561.794520:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489561.794527:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489561.794532:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423400 00000400:00000200:3.0:1782489561.794670:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489561.794679:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c35 [1] + 224 00000800:00000001:3.0:1782489561.794689:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489561.794699:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489561.794703:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489561.794727:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489561.794737:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489561.794749:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489561.794762:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.794821:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.794827:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489561.794847:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489561.794860:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489561.794863:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489561.796912:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.796954:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489561.796958:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.796961:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.796980:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.796990:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489561.797001:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.797006:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.797013:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.797017:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423480 00000400:00000200:2.0:1782489561.797024:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489561.797032:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c4d [1] + 224 00000800:00000001:2.0:1782489561.797049:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.797058:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489561.797062:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489561.797068:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489561.797077:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489561.797088:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489561.797101:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489561.797110:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489561.797120:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489561.797123:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:0.0:1782489561.797737:0:2417:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489561.797743:0:2417:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489561.797748:0:2417:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a058. 00000400:00000200:0.0:1782489561.797755:0:2417:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724dbb0 00000400:00000010:0.0:1782489561.797759:0:2417:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724dbb0. 00000100:00000001:0.0:1782489561.797764:0:2417:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489561.797773:0:2417:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489561.797785:0:2417:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.797793:0:2417:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489561.797796:0:2417:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.797799:0:2417:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489561.797804:0:2417:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489561.797808:0:2417:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489561.797811:0:2417:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.797815:0:2417:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489561.797828:0:2417:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc3460 time=42 v=60 (60 60 60 60) 00000100:00001000:0.0:1782489561.797834:0:2417:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3418 time=78 v=5 (1 1 1 1) 00000100:00000001:0.0:1782489561.797837:0:2417:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489561.797839:0:2417:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489561.797843:0:2417:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1782489561.797865:0:2417:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.797870:0:2417:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489561.797872:0:2417:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ce5830d8800: skip recheck: last_committed 30064771619 00000100:00000001:0.0:1782489561.797875:0:2417:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489561.797876:0:2417:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.797883:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.797890:0:2417:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.797895:0:2417:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489561.797897:0:2417:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489561.797901:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489561.797909:0:2417:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5a621c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684544:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.797914:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.797918:0:2417:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489561.797921:0:2417:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5a621c700 x1869066440684544/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.797928:0:2417:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489561.797932:0:2417:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489561.797936:0:2417:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677d200. 02000000:00000001:0.0:1782489561.797942:0:2417:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489561.797945:0:2417:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489561.797948:0:2417:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3000 refcount=8 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000020:00000001:0.0:1782489561.797950:0:2417:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489561.797953:0:2417:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5ba198700. 02000000:00000001:0.0:1782489561.797957:0:2417:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489561.797960:0:2417:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489561.797962:0:2417:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5a621c700. 00000100:00000001:0.0:1782489561.797966:0:2417:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489561.797968:0:2417:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.797973:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.797975:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.797984:0:2417:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1782489561.797988:0:2417:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1782489561.797989:0:2417:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1782489561.797991:0:2417:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5b687a420. 00000400:00000200:0.0:1782489561.797993:0:2417:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5b724da18 00000400:00000010:0.0:1782489561.797994:0:2417:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5b724da18. 00000100:00000001:0.0:1782489561.797997:0:2417:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1782489561.798001:0:2417:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1782489561.798009:0:2417:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.798015:0:2417:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1782489561.798017:0:2417:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.798018:0:2417:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1782489561.798020:0:2417:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1782489561.798021:0:2417:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1782489561.798023:0:2417:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1782489561.798025:0:2417:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1782489561.798030:0:2417:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc1460 time=42 v=60 (60 60 60 60) 00000100:00001000:0.0:1782489561.798034:0:2417:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc1418 time=66 v=5 (1 1 1 1) 00000100:00000001:0.0:1782489561.798036:0:2417:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1782489561.798037:0:2417:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1782489561.798039:0:2417:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1782489561.798043:0:2417:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1782489561.798049:0:2417:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.798051:0:2417:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1782489561.798053:0:2417:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ce5830d8800: skip recheck: last_committed 21474836540 00000100:00000001:0.0:1782489561.798055:0:2417:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1782489561.798056:0:2417:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1782489561.798059:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.798065:0:2417:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1782489561.798073:0:2417:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1782489561.798074:0:2417:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1782489561.798079:0:2417:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1782489561.798086:0:2417:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5a621c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2417:1869066440684672:192.168.202.139@tcp:400:kworker.0 00000100:00000001:0.0:1782489561.798090:0:2417:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.798092:0:2417:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1782489561.798095:0:2417:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5a621c000 x1869066440684672/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489646 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1782489561.798101:0:2417:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1782489561.798103:0:2417:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1782489561.798105:0:2417:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce58677c000. 02000000:00000001:0.0:1782489561.798108:0:2417:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1782489561.798110:0:2417:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1782489561.798111:0:2417:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc1000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000020:00000001:0.0:1782489561.798113:0:2417:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1782489561.798115:0:2417:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5ba198100. 02000000:00000001:0.0:1782489561.798117:0:2417:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1782489561.798118:0:2417:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1782489561.798120:0:2417:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5a621c000. 00000100:00000001:0.0:1782489561.798122:0:2417:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1782489561.798124:0:2417:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.798125:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1782489561.798135:0:2417:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1782489561.798138:0:2417:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1782489561.798140:0:2417:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1782489561.798142:0:2417:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489562.285177:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489562.285187:0:213139:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:3.0:1782489562.285189:0:213139:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:3.0:1782489562.285194:0:213139:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489562.285200:0:213139:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:3.0:1782489562.285202:0:213139:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489562.285205:0:213139:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489562.285207:0:213139:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489562.285211:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489562.285224:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489562.285314:0:238718:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489562.285319:0:238718:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489562.285323:0:238718:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489563.309824:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489563.309831:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489563.309836:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489563.309837:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1782489563.310327:0:254078:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1782489563.310331:0:254078:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1782489563.310335:0:254078:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:1.0:1782489564.122721:0:286519:0:(llite_lib.c:4108:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489564.334534:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489564.334541:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489564.334546:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489564.334547:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489564.334748:0:244254:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489564.334811:0:244254:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489564.334815:0:244254:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489565.355567:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489565.355574:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489565.355579:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489565.355581:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1782489565.355663:0:238717:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1782489565.355667:0:238717:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1782489565.355670:0:238717:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1782489565.611575:0:213139:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304330 (cpu 0) has reached its deadline 4303810247, at 4303810304 00000100:00000010:3.0:1782489565.611591:0:213139:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce587231ae0. 00000100:00000040:2.0:1782489565.611683:0:237024:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304220 (cpu 0) has reached its deadline 4303810247, at 4303810304 00000100:00000010:2.0:1782489565.611689:0:237024:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce587231960. 00010000:00000001:3.0:1782489566.380510:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489566.380524:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489566.380531:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489566.380532:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489566.380937:0:249964:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489566.380941:0:249964:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489566.380945:0:249964:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1782489566.891786:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) dd22dfa7-56fe-4502-9376-64e8b3112148->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1782489566.891797:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1782489566.891810:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5babe0a80. 00000020:00000040:2.0:1782489566.891817:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5834b1800 refcount=4 obd=MGC192.168.202.139@tcp 00000100:00000001:2.0:1782489566.891835:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1782489566.891839:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1782489566.891844:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1782489566.891919:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8602400. 00000100:00000001:2.0:1782489566.891927:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.891939:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging dd22dfa7-56fe-4502-9376-64e8b3112148->MGS req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1782489566.891950:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1782489566.891954:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.891960:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a336e37f] to pc [ptlrpcd_01_01+1] req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.892021:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.892026:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.892030:0:2419:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1782489566.892037:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.892045:0:2419:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1782489566.892047:0:2419:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.892051:0:2419:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489566.892054:0:2419:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489566.892057:0:2419:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.892061:0:2419:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5babe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2419:1869066440684928:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.892065:0:2419:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1782489566.892071:0:2419:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1782489566.892074:0:2419:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.892077:0:2419:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1782489566.892083:0:2419:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fac600. 02000000:00000001:3.0:1782489566.892085:0:2419:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489566.892092:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:3.0:1782489566.892096:0:2419:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9b00. 00000400:00000010:3.0:1782489566.892104:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09ab28. 00000100:00000200:3.0:1782489566.892111:0:2419:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440684928, portal 25 00000100:00000001:3.0:1782489566.892114:0:2419:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489566.892115:0:2419:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109227301504 : -108964482250112 : ffff9ce5babe0a80) 00000100:00000040:3.0:1782489566.892121:0:2419:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.892128:0:2419:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489566.892131:0:2419:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:3.0:1782489566.892134:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09ad48. 00000100:00000200:3.0:1782489566.892136:0:2419:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1869066440684928, offset 0 00000400:00000200:3.0:1782489566.892144:0:2419:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489566.892154:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.892170:0:2419:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489566.892186:0:2419:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1782489566.892191:0:2419:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489566.892195:0:2419:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1782489566.892198:0:2419:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489566.892209:0:2419:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536810:536810:256:4294967295] 192.168.202.139@tcp LPNI seq info [536777:536777:8:4294967295] 00000400:00000200:3.0:1782489566.892214:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.892223:0:2419:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:3.0:1782489566.892229:0:2419:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489566.892235:0:2419:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9700. 00000800:00000200:3.0:1782489566.892242:0:2419:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:3.0:1782489566.892251:0:2419:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:3.0:1782489566.892255:0:2419:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1782489566.892269:0:2419:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489566.892273:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489566.892276:0:2419:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 2 is 4303815360 00000100:00000010:3.0:1782489566.892280:0:2419:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce5b8b68180. 00000100:00000001:3.0:1782489566.892288:0:2419:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1782489566.892295:0:2419:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 3 is 4303815360 00000100:00000010:3.0:1782489566.892298:0:2419:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce5b8b689c0. 00000100:00000001:3.0:1782489566.892300:0:2419:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1782489566.892303:0:2419:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.892306:0:2419:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.892309:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.892311:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.892337:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.892341:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1782489566.892344:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1782489566.892347:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5babe0700. 00000020:00000040:2.0:1782489566.892349:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3000 refcount=9 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000100:00000001:2.0:1782489566.892351:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1782489566.892353:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1782489566.892355:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1782489566.892359:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8602f00. 00000100:00000001:2.0:1782489566.892362:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.892367:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1782489566.892374:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1782489566.892377:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.892381:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000db37cce5] to pc [ptlrpcd_01_00+0] req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.894469:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489566.894475:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1782489566.894481:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.894483:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.894487:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.894490:0:2419:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:3.0:1782489566.894492:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.894511:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489566.894513:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.894515:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.894516:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.894518:0:2419:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1782489566.894531:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.894541:0:2419:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1782489566.894542:0:2419:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.894547:0:2419:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489566.894550:0:2419:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489566.894552:0:2419:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.894556:0:2419:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5babe0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2419:1869066440685056:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.894561:0:2419:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1782489566.894567:0:2419:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1782489566.894570:0:2419:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.894573:0:2419:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1782489566.894579:0:2419:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fada00. 02000000:00000001:3.0:1782489566.894581:0:2419:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489566.894588:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:3.0:1782489566.894593:0:2419:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9a50. 00000400:00000010:3.0:1782489566.894600:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09acc0. 00000100:00000200:3.0:1782489566.894607:0:2419:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685056, portal 10 00000100:00000001:3.0:1782489566.894609:0:2419:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489566.894610:0:2419:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109227300608 : -108964482251008 : ffff9ce5babe0700) 00000100:00000040:3.0:1782489566.895337:0:2419:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.895347:0:2419:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489566.895350:0:2419:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:3.0:1782489566.895355:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09a880. 00000100:00000200:3.0:1782489566.895359:0:2419:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440685056, offset 0 00000400:00000200:3.0:1782489566.895366:0:2419:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489566.895375:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.895391:0:2419:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489566.895396:0:2419:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1782489566.895400:0:2419:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489566.895404:0:2419:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1782489566.895408:0:2419:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489566.895411:0:2419:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536811:536811:255:4294967295] 192.168.202.139@tcp LPNI seq info [536778:536778:7:4294967295] 00000400:00000200:3.0:1782489566.895415:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.895425:0:2419:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000100:00000001:2.0:1782489566.895841:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.895845:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1782489566.895850:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000800:00000200:3.0:1782489566.895936:0:2419:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489566.895943:0:2419:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9500. 00000800:00000200:3.0:1782489566.895949:0:2419:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:3.0:1782489566.895957:0:2419:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:3.0:1782489566.895961:0:2419:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1782489566.895965:0:2419:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.895970:0:2419:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: using a timeout of 25128 usecs (26 jiffies) 00000400:00000200:3.0:1782489566.895974:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489566.895977:0:2419:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303811614 00000100:00000001:3.0:1782489566.895980:0:2419:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.895983:0:2419:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303811614 00000100:00000001:3.0:1782489566.895985:0:2419:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.895987:0:2419:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.895989:0:2419:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.895992:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.895996:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.895998:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.896000:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.896001:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.896003:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1782489566.896058:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1782489566.896061:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9700. 00000400:00000200:3.0:1782489566.896068:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489566.896079:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:3.0:1782489566.896084:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09ad48 00000400:00000010:3.0:1782489566.896086:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09ad48. 00000100:00000001:3.0:1782489566.896091:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1782489566.896097:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.896107:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489566.896111:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.896118:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.896119:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1782489566.899220:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1782489566.899230:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9500. 00000400:00000200:3.0:1782489566.899237:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489566.899246:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:3.0:1782489566.899253:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09a880 00000400:00000010:3.0:1782489566.899256:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09a880. 00000100:00000001:3.0:1782489566.899261:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1782489566.899269:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899284:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489566.899287:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899296:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899297:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1782489566.899307:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.899319:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489566.899322:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.899324:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489566.899330:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.899335:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489566.899341:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.899345:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.899368:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.899372:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423600 00000400:00000200:3.0:1782489566.899377:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.899383:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c65 [1] + 224 00000800:00000001:3.0:1782489566.899391:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.899398:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489566.899400:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489566.899403:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489566.899408:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489566.899412:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489566.899421:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899452:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489566.899459:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.899461:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.899477:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.899481:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899490:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.899493:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1782489566.899498:0:2419:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489566.899500:0:2419:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489566.899503:0:2419:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9a50. 00000400:00000200:3.0:1782489566.899506:0:2419:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09acc0 00000400:00000010:3.0:1782489566.899508:0:2419:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09acc0. 00000100:00000001:3.0:1782489566.899510:0:2419:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489566.899515:0:2419:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489566.899523:0:2419:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899530:0:2419:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489566.899531:0:2419:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.899539:0:2419:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489566.899542:0:2419:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489566.899544:0:2419:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489566.899547:0:2419:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.899549:0:2419:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489566.899558:0:2419:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc3460 time=47 v=60 (60 60 60 60) 00000100:00001000:3.0:1782489566.899562:0:2419:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3418 time=83 v=5 (1 1 1 1) 00000100:00000001:3.0:1782489566.899565:0:2419:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489566.899567:0:2419:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489566.899570:0:2419:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1782489566.899573:0:2419:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899577:0:2419:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489566.899579:0:2419:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ce5830d8800: skip recheck: last_committed 30064771619 00000100:00000001:3.0:1782489566.899581:0:2419:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489566.899583:0:2419:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.899588:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899595:0:2419:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.899598:0:2419:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489566.899600:0:2419:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489566.899604:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489566.899612:0:2419:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5babe0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2419:1869066440685056:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.899617:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899619:0:2419:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489566.899622:0:2419:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5babe0700 x1869066440685056/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.899629:0:2419:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489566.899631:0:2419:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489566.899648:0:2419:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fada00. 02000000:00000001:3.0:1782489566.899652:0:2419:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489566.899655:0:2419:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489566.899657:0:2419:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3000 refcount=8 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000020:00000001:3.0:1782489566.899663:0:2419:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489566.899666:0:2419:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8602f00. 02000000:00000001:3.0:1782489566.899671:0:2419:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489566.899673:0:2419:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489566.899675:0:2419:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5babe0700. 00000100:00000001:3.0:1782489566.899678:0:2419:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489566.899679:0:2419:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.899681:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899683:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.899684:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.899686:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.899688:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:2.0:1782489566.900916:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5babe0000. 00000020:00000040:2.0:1782489566.900922:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc1000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000100:00000001:2.0:1782489566.900926:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1782489566.900929:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1782489566.900933:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1782489566.900941:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8602900. 00000100:00000001:2.0:1782489566.900948:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.900957:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1782489566.900966:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1782489566.900971:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.900977:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e62ca981] to pc [ptlrpcd_01_01+1] req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.901002:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.901005:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1782489566.901010:0:284611:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1782489566.901013:0:284611:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5babe3b80. 00000020:00000040:2.0:1782489566.901015:0:284611:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3800 refcount=4 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000100:00000001:2.0:1782489566.901018:0:284611:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1782489566.901019:0:284611:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1782489566.901021:0:284611:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1782489566.901024:0:284611:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8602d00. 00000100:00000001:2.0:1782489566.901026:0:284611:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.901030:0:284611:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1782489566.901036:0:284611:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1782489566.901038:0:284611:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.901042:0:284611:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000060395ecf] to pc [ptlrpcd_01_00+0] req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.901050:0:284611:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.901061:0:284611:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1782489566.901064:0:284611:0:(pinger.c:249:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1782489566.901070:0:284611:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (9149) 00000100:00000001:3.0:1782489566.901371:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.901375:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.901378:0:2418:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1782489566.901384:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.901392:0:2418:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1782489566.901395:0:2418:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.901399:0:2418:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489566.901401:0:2418:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489566.901404:0:2418:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.901407:0:2418:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5babe3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685312:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.901411:0:2418:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1782489566.901416:0:2418:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1782489566.901419:0:2418:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.901422:0:2418:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1782489566.901426:0:2418:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fada00. 02000000:00000001:3.0:1782489566.901453:0:2418:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489566.901460:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:3.0:1782489566.901465:0:2418:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9a50. 00000400:00000010:3.0:1782489566.901471:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09acc0. 00000100:00000200:3.0:1782489566.901477:0:2418:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685312, portal 4 00000100:00000001:3.0:1782489566.901480:0:2418:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489566.901481:0:2418:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109227314048 : -108964482237568 : ffff9ce5babe3b80) 00000100:00000040:3.0:1782489566.901495:0:2418:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.901503:0:2418:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489566.901505:0:2418:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:3.0:1782489566.901508:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09a880. 00000100:00000200:3.0:1782489566.901511:0:2418:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1869066440685312, offset 0 00000400:00000200:3.0:1782489566.901517:0:2418:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489566.901525:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.901531:0:2418:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489566.901537:0:2418:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1782489566.901540:0:2418:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489566.901544:0:2418:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1782489566.901547:0:2418:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489566.901551:0:2418:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536812:536812:256:4294967295] 192.168.202.139@tcp LPNI seq info [536779:536779:8:4294967295] 00000400:00000200:3.0:1782489566.901555:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.901564:0:2418:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:3.0:1782489566.901570:0:2418:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489566.901574:0:2418:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9500. 00000800:00000200:3.0:1782489566.901580:0:2418:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:3.0:1782489566.901587:0:2418:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:3.0:1782489566.901591:0:2418:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1782489566.901610:0:2418:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.901614:0:2418:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: using a timeout of 92002 usecs (93 jiffies) 00000400:00000200:3.0:1782489566.901625:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489566.901627:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303811687 00000100:00000001:3.0:1782489566.901630:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.901633:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303811687 00000100:00000001:3.0:1782489566.902790:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.902799:0:2418:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.902816:0:2418:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.902820:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.902823:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.902830:0:2418:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489566.902832:0:2418:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1782489566.902835:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.902837:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.902838:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.902841:0:2418:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1782489566.902843:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.902845:0:2418:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489566.902846:0:2418:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.902848:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.902849:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.902850:0:2418:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1782489566.902929:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.902938:0:2418:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1782489566.902940:0:2418:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.902944:0:2418:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489566.902947:0:2418:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489566.902949:0:2418:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489566.902952:0:2418:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5babe0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685184:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.902956:0:2418:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1782489566.902961:0:2418:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1782489566.902964:0:2418:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.902966:0:2418:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1782489566.902971:0:2418:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fadc00. 02000000:00000001:3.0:1782489566.902973:0:2418:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1782489566.902978:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:3.0:1782489566.902982:0:2418:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9aa8. 00000400:00000010:3.0:1782489566.902987:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09ad48. 00000100:00000200:3.0:1782489566.902993:0:2418:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685184, portal 10 00000100:00000001:3.0:1782489566.902996:0:2418:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489566.902997:0:2418:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109227298816 : -108964482252800 : ffff9ce5babe0000) 00000100:00000040:3.0:1782489566.903012:0:2418:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.903019:0:2418:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489566.903021:0:2418:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:3.0:1782489566.903024:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09aaa0. 00000100:00000200:3.0:1782489566.903027:0:2418:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440685184, offset 0 00000400:00000200:3.0:1782489566.903034:0:2418:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489566.903040:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.903046:0:2418:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489566.903050:0:2418:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1782489566.903053:0:2418:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489566.903057:0:2418:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1782489566.903059:0:2418:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489566.903062:0:2418:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536813:536813:255:4294967295] 192.168.202.139@tcp LPNI seq info [536780:536780:7:4294967295] 00000400:00000200:3.0:1782489566.903066:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.903077:0:2418:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:3.0:1782489566.903082:0:2418:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489566.903086:0:2418:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9700. 00000800:00000200:3.0:1782489566.903091:0:2418:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000800:00000200:3.0:1782489566.903098:0:2418:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000800:00000200:3.0:1782489566.903101:0:2418:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1782489566.903105:0:2418:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.903109:0:2418:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: using a timeout of 25113 usecs (26 jiffies) 00000400:00000200:3.0:1782489566.903112:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489566.903114:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303811621 00000100:00000001:3.0:1782489566.903116:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.903119:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303811621 00000100:00000001:3.0:1782489566.903121:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303815360 00000100:00000001:3.0:1782489566.903123:0:2418:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.903130:0:2418:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.903132:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.903135:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.903157:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.903159:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489566.903160:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.903162:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1782489566.903259:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489566.903264:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9500. 00000400:00000200:2.0:1782489566.903270:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489566.903275:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489566.903280:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09a880 00000400:00000010:2.0:1782489566.903282:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09a880. 00000100:00000001:2.0:1782489566.903286:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489566.903291:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.903301:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489566.903305:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.903312:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489566.903314:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489566.903375:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489566.903378:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9700. 00000400:00000200:2.0:1782489566.903380:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489566.903384:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489566.903387:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09aaa0 00000400:00000010:2.0:1782489566.903389:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09aaa0. 00000100:00000001:2.0:1782489566.903391:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489566.903394:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.903401:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489566.903404:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.903411:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489566.903412:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1782489566.906215:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.906235:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1782489566.906239:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.906242:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489566.906251:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.906259:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:3.0:1782489566.906269:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.906274:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.906280:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.906284:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 25 MB=0x6a3e7f1423580 00000400:00000200:3.0:1782489566.906289:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489566.906295:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c55 [1] + 224 00000800:00000001:3.0:1782489566.906303:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.906313:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1782489566.906316:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1782489566.906321:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:3.0:1782489566.906328:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489566.906337:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489566.906348:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.906375:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1782489566.906384:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1782489566.906387:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.906527:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1782489566.906533:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1782489566.906541:0:2419:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1782489566.906544:0:2419:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1782489566.906548:0:2419:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9b00. 00000400:00000200:2.0:1782489566.906554:0:2419:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09ab28 00000400:00000010:2.0:1782489566.906556:0:2419:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09ab28. 00000100:00000001:2.0:1782489566.906560:0:2419:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489566.906565:0:2419:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1782489566.906574:0:2419:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.906581:0:2419:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1782489566.906583:0:2419:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.906587:0:2419:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:2.0:1782489566.906590:0:2419:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1782489566.906592:0:2419:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1782489566.906594:0:2419:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1782489566.906597:0:2419:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1782489566.906604:0:2419:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5834b1c60 time=47 v=5 (5 5 5 5) 00000100:00001000:2.0:1782489566.906607:0:2419:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5834b1c18 time=83 v=5 (1 1 1 1) 00000100:00000001:2.0:1782489566.906610:0:2419:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1782489566.906612:0:2419:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1782489566.906614:0:2419:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1782489566.906616:0:2419:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489566.906620:0:2419:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489566.906624:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.906631:0:2419:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1782489566.906644:0:2419:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1782489566.906647:0:2419:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1782489566.906652:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1782489566.906668:0:2419:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5babe0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2419:1869066440684928:192.168.202.139@tcp:400:kworker.0 00000100:00000001:2.0:1782489566.906693:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.906696:0:2419:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489566.906700:0:2419:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5babe0a80 x1869066440684928/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.906707:0:2419:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1782489566.906711:0:2419:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1782489566.906715:0:2419:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fac600. 02000000:00000001:2.0:1782489566.906719:0:2419:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1782489566.906722:0:2419:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:2.0:1782489566.906724:0:2419:0:(genops.c:1244:class_import_put()) import ffff9ce5834b1800 refcount=3 obd=MGC192.168.202.139@tcp 00000020:00000001:2.0:1782489566.906726:0:2419:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:2.0:1782489566.906729:0:2419:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8602400. 02000000:00000001:2.0:1782489566.906733:0:2419:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1782489566.906734:0:2419:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1782489566.906736:0:2419:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5babe0a80. 00000100:00000001:2.0:1782489566.906739:0:2419:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1782489566.906741:0:2419:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.906743:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489566.906751:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1782489566.906753:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489566.906755:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1782489566.906757:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1782489566.909047:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.909064:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489566.909068:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.909072:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489566.909082:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.909089:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489566.909098:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.909102:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.909108:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.909112:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423680 00000400:00000200:2.0:1782489566.909116:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.909122:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c85 [1] + 224 00000800:00000001:2.0:1782489566.909128:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.909397:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489566.909401:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489566.909406:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489566.909413:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489566.909423:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489566.909464:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.909477:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489566.909485:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.909488:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.915318:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.915327:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1782489566.915335:0:2418:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489566.915343:0:2418:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489566.915346:0:2418:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9aa8. 00000400:00000200:3.0:1782489566.915353:0:2418:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09ad48 00000400:00000010:3.0:1782489566.915355:0:2418:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09ad48. 00000100:00000001:3.0:1782489566.915360:0:2418:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489566.915371:0:2418:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489566.915383:0:2418:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.915391:0:2418:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489566.915393:0:2418:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.915396:0:2418:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489566.915399:0:2418:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489566.915402:0:2418:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489566.915404:0:2418:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.915407:0:2418:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489566.915417:0:2418:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc1460 time=47 v=60 (60 60 60 60) 00000100:00001000:3.0:1782489566.915421:0:2418:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc1418 time=71 v=5 (1 1 1 1) 00000100:00000001:3.0:1782489566.915424:0:2418:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489566.915426:0:2418:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1782489566.917258:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.917280:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489566.917285:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.917288:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489566.917297:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.917305:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489566.917314:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.917319:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.917325:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.917330:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 4 MB=0x6a3e7f1423700 00000400:00000200:2.0:1782489566.917335:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489566.917340:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c75 [1] + 224 00000800:00000001:2.0:1782489566.917348:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.917359:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489566.917363:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489566.917367:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489566.917374:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489566.917383:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489566.917395:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489566.917424:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:3.0:1782489566.919244:0:2418:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1782489566.919263:0:2418:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1782489566.919275:0:2418:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.919281:0:2418:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489566.919283:0:2418:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ce5830d8800: skip recheck: last_committed 21474836540 00000100:00000001:3.0:1782489566.919285:0:2418:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489566.919287:0:2418:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.919292:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.919300:0:2418:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.919304:0:2418:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489566.919306:0:2418:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489566.919318:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489566.919326:0:2418:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5babe0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685184:192.168.202.139@tcp:400:kworker.0 00000400:00000001:3.0:1782489566.919336:0:2418:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489566.919338:0:2418:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489566.919341:0:2418:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9a50. 00000400:00000200:3.0:1782489566.919344:0:2418:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09acc0 00000400:00000010:3.0:1782489566.919347:0:2418:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09acc0. 00000100:00000001:3.0:1782489566.919351:0:2418:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489566.919357:0:2418:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489566.919366:0:2418:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.919372:0:2418:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489566.919374:0:2418:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.919376:0:2418:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489566.919380:0:2418:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489566.919382:0:2418:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489566.919384:0:2418:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489566.919388:0:2418:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489566.919396:0:2418:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5b6bc3c60 time=47 v=5 (5 5 5 5) 00000100:00001000:3.0:1782489566.919400:0:2418:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3c18 time=124 v=5 (1 1 0 0) 00000100:00000001:3.0:1782489566.919403:0:2418:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489566.919404:0:2418:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489566.919406:0:2418:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1782489566.919408:0:2418:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.919410:0:2418:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489566.919412:0:2418:0:(client.c:3107:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ce5830d8800: skip recheck: last_committed 85899368364 00000100:00000001:3.0:1782489566.919414:0:2418:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489566.919415:0:2418:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489566.919419:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.919455:0:2418:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489566.919458:0:2418:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489566.919459:0:2418:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489566.919463:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489566.919471:0:2418:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5babe3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685312:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489566.919476:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.919479:0:2418:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489566.919482:0:2418:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5babe0000 x1869066440685184/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489651 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.919489:0:2418:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489566.919492:0:2418:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489566.919495:0:2418:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fadc00. 02000000:00000001:3.0:1782489566.919499:0:2418:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489566.919501:0:2418:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489566.919503:0:2418:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc1000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000020:00000001:3.0:1782489566.919517:0:2418:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489566.919520:0:2418:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8602900. 02000000:00000001:3.0:1782489566.919524:0:2418:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489566.919526:0:2418:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489566.919528:0:2418:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5babe0000. 00000100:00000001:3.0:1782489566.919531:0:2418:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489566.919532:0:2418:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.919534:0:2418:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489566.919538:0:2418:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5babe3b80 x1869066440685312/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489582 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489566.919544:0:2418:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489566.919545:0:2418:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489566.919547:0:2418:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fada00. 02000000:00000001:3.0:1782489566.919549:0:2418:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489566.919550:0:2418:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489566.919556:0:2418:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3800 refcount=3 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000020:00000001:3.0:1782489566.919558:0:2418:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489566.919560:0:2418:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8602d00. 02000000:00000001:3.0:1782489566.919562:0:2418:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489566.919563:0:2418:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489566.919565:0:2418:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5babe3b80. 00000100:00000001:3.0:1782489566.919568:0:2418:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489566.919569:0:2418:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.919572:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489566.919580:0:2418:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489566.919581:0:2418:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489566.919584:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489566.919587:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1782489566.920536:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489566.920546:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:3.0:1782489567.403522:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489567.403529:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489567.403534:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489567.403536:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489567.403598:0:238718:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489567.403601:0:238718:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489567.403603:0:238718:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489568.427566:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489568.427573:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489568.427578:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489568.427579:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1782489568.427729:0:254078:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1782489568.427732:0:254078:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1782489568.427735:0:254078:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489569.452941:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489569.452948:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489569.452952:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489569.452952:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489569.454695:0:244254:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489569.454701:0:244254:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489569.454707:0:244254:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:3.0:1782489570.475932:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489570.475942:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489570.475950:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489570.475954:0:213139:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:3.0:1782489570.475957:0:213139:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:3.0:1782489570.475965:0:213139:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:3.0:1782489570.475967:0:213139:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:3.0:1782489570.475969:0:213139:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489570.475972:0:213139:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:3.0:1782489570.475985:0:213139:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489570.475991:0:213139:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489570.475993:0:213139:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1782489570.479004:0:238717:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1782489570.479038:0:238717:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1782489570.479045:0:238717:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1782489570.732743:0:213139:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304330 (cpu 0) has reached its deadline 4303815360, at 4303815424 00000100:00000010:3.0:1782489570.732764:0:213139:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce5b8b689c0. 00000100:00000040:2.0:1782489570.733830:0:237024:0:(connection.c:75:cpu_latency_work()) work item of ffff9ce585304220 (cpu 0) has reached its deadline 4303815360, at 4303815426 00000100:00000010:2.0:1782489570.733844:0:237024:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9ce5b8b68180. 00010000:00000001:3.0:1782489571.499705:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489571.499715:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489571.499722:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489571.499723:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1782489571.499800:0:249964:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1782489571.499805:0:249964:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1782489571.499808:0:249964:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1782489572.011671:0:286401:0:(pinger.c:231:ptlrpc_pinger_process_import()) dd22dfa7-56fe-4502-9376-64e8b3112148->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1782489572.011684:0:286401:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1782489572.011693:0:286401:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5b6bfbb80. 00000020:00000040:3.0:1782489572.011699:0:286401:0:(genops.c:1231:class_import_get()) import ffff9ce5834b1800 refcount=4 obd=MGC192.168.202.139@tcp 00000100:00000001:3.0:1782489572.011704:0:286401:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1782489572.011707:0:286401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1782489572.011712:0:286401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1782489572.011719:0:286401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8ba9900. 00000100:00000001:3.0:1782489572.011726:0:286401:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011738:0:286401:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging dd22dfa7-56fe-4502-9376-64e8b3112148->MGS req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1782489572.011749:0:286401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1782489572.011753:0:286401:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011760:0:286401:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ba84cd04] to pc [ptlrpcd_01_01+1] req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.011790:0:286401:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011793:0:286401:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1782489572.011797:0:286401:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1782489572.011799:0:286401:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5b6bfb800. 00000020:00000040:3.0:1782489572.011802:0:286401:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3000 refcount=9 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000100:00000001:3.0:1782489572.011804:0:286401:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1782489572.011805:0:286401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1782489572.011807:0:286401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1782489572.011810:0:286401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8ba9400. 00000100:00000001:3.0:1782489572.011829:0:286401:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011834:0:286401:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0000_UUID req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1782489572.011841:0:286401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1782489572.011843:0:286401:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011847:0:286401:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c2a705e8] to pc [ptlrpcd_01_00+0] req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.011855:0:286401:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011858:0:286401:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1782489572.011861:0:286401:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1782489572.011864:0:286401:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5b6bf9880. 00000020:00000040:3.0:1782489572.011866:0:286401:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc1000 refcount=4 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000100:00000001:3.0:1782489572.011868:0:286401:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1782489572.011870:0:286401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1782489572.011871:0:286401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1782489572.011874:0:286401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8ba9100. 00000100:00000001:3.0:1782489572.011877:0:286401:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011881:0:286401:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-MDT0001_UUID req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1782489572.011888:0:286401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1782489572.011888:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.011890:0:286401:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.011892:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1782489572.011893:0:286401:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003e322d65] to pc [ptlrpcd_01_01+1] req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.011894:0:2419:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000001:3.0:1782489572.011901:0:286401:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011903:0:286401:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1782489572.011906:0:286401:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1782489572.011908:0:286401:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9ce5b6bfa680. 00000020:00000040:3.0:1782489572.011910:0:286401:0:(genops.c:1231:class_import_get()) import ffff9ce5b6bc3800 refcount=4 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000100:00000001:3.0:1782489572.011912:0:286401:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1782489572.011919:0:286401:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1782489572.011920:0:286401:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1782489572.011925:0:286401:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9ce5b8ba9700. 00000100:00000001:3.0:1782489572.011928:0:286401:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011932:0:286401:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0000_UUID req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1782489572.011939:0:286401:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1782489572.011941:0:286401:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011945:0:286401:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d44bd837] to pc [ptlrpcd_01_00+0] req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.011950:0:286401:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.011952:0:286401:0:(pinger.c:231:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1782489572.011955:0:286401:0:(pinger.c:249:ptlrpc_pinger_process_import()) bf41b9ca-a66c-4f13-9d79-fff1432569da->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1782489572.011966:0:286401:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (9154) 00000100:00000001:3.0:1782489572.011977:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.011980:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489572.011981:0:2418:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1782489572.011986:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.011993:0:2418:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1782489572.011995:0:2418:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489572.011996:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1782489572.011998:0:2418:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489572.012001:0:2418:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489572.012003:0:2418:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012003:0:2419:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1782489572.012004:0:2419:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1782489572.012006:0:2418:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5b6bfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685568:192.168.202.139@tcp:400:kworker.0 02000000:00000001:2.0:1782489572.012006:0:2419:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1782489572.012007:0:2419:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1782489572.012009:0:2419:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012011:0:2418:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00100000:2.0:1782489572.012012:0:2419:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5b6bfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2419:1869066440685440:192.168.202.139@tcp:400:kworker.0 00000100:00000001:2.0:1782489572.012014:0:2419:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1782489572.012016:0:2418:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1782489572.012016:0:2419:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1782489572.012019:0:2418:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1782489572.012019:0:2419:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.012021:0:2418:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1782489572.012021:0:2419:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1782489572.012026:0:2418:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fa6200. 02000000:00000010:2.0:1782489572.012026:0:2419:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fac800. 02000000:00000001:2.0:1782489572.012028:0:2419:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.012029:0:2418:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1782489572.012035:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489572.012036:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:3.0:1782489572.012039:0:2418:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9a50. 00000400:00000010:2.0:1782489572.012039:0:2419:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce599649d10. 00000400:00000010:3.0:1782489572.012046:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09acc0. 00000400:00000010:2.0:1782489572.012046:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce588d97088. 00000100:00000200:2.0:1782489572.012054:0:2419:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685440, portal 25 00000100:00000200:3.0:1782489572.012056:0:2418:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685568, portal 10 00000100:00000001:2.0:1782489572.012056:0:2419:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489572.012057:0:2418:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1782489572.012058:0:2419:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109160303488 : -108964549248128 : ffff9ce5b6bfbb80) 00000100:00000001:3.0:1782489572.012059:0:2418:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109160302592 : -108964549249024 : ffff9ce5b6bfb800) 00000100:00000040:2.0:1782489572.012063:0:2419:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1782489572.012066:0:2418:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.012074:0:2418:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:2.0:1782489572.012075:0:2419:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489572.012076:0:2418:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000100:00000040:2.0:1782489572.012077:0:2419:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:3.0:1782489572.012079:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09ad48. 00000400:00000010:2.0:1782489572.012079:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce588d97dd0. 00000100:00000200:3.0:1782489572.012082:0:2418:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440685568, offset 0 00000100:00000200:2.0:1782489572.012082:0:2419:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1869066440685440, offset 0 00000400:00000200:2.0:1782489572.012088:0:2419:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489572.012093:0:2418:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:2.0:1782489572.012096:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.012103:0:2419:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:2.0:1782489572.012109:0:2419:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1782489572.012112:0:2419:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:2.0:1782489572.012116:0:2419:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1782489572.012119:0:2419:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:2.0:1782489572.012122:0:2419:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536814:536814:256:4294967295] 192.168.202.139@tcp LPNI seq info [536781:536781:8:4294967295] 00000400:00000200:2.0:1782489572.012126:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.012144:0:2419:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000400:00000200:3.0:1782489572.012149:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000800:00000200:2.0:1782489572.012149:0:2419:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489572.012152:0:2418:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489572.012154:0:2418:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000800:00000010:2.0:1782489572.012154:0:2419:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8602c00. 00000400:00000200:3.0:1782489572.012156:0:2418:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489572.012159:0:2418:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000800:00000200:2.0:1782489572.012160:0:2419:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000400:00000200:3.0:1782489572.012161:0:2418:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489572.012163:0:2418:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536815:536815:255:4294967295] 192.168.202.139@tcp LPNI seq info [536782:536782:7:4294967295] 00000400:00000200:3.0:1782489572.012166:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000800:00000200:2.0:1782489572.012168:0:2419:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000400:00000200:3.0:1782489572.012171:0:2418:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:2.0:1782489572.012172:0:2419:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8602c00 type 1, nob 320 niov 1 nkiov 1 00000800:00000200:3.0:1782489572.012175:0:2418:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000800:00000010:3.0:1782489572.012178:0:2418:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9500. 00000800:00000200:3.0:1782489572.012181:0:2418:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000100:00000001:2.0:1782489572.012183:0:2419:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1782489572.012185:0:2418:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000400:00000200:2.0:1782489572.012187:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000800:00000200:3.0:1782489572.012188:0:2418:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1782489572.012189:0:2419:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 2 is 4303820480 00000100:00000001:3.0:1782489572.012190:0:2418:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.012193:0:2418:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: using a timeout of 25123 usecs (26 jiffies) 00000100:00000010:2.0:1782489572.012194:0:2419:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce582ef0840. 00000400:00000200:3.0:1782489572.012196:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489572.012197:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303816730 00000100:00000001:2.0:1782489572.012200:0:2419:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 2 00000100:00000001:2.0:1782489572.012205:0:2419:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS new deadline estimation for cpu 3 is 4303820480 00000100:00000010:2.0:1782489572.012208:0:2419:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9ce582ef0d20. 00000100:00000001:2.0:1782489572.012211:0:2419:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.139@tcp: PM QoS request now active for cpu 3 00000100:00000001:2.0:1782489572.012214:0:2419:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.012215:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303820480 00000100:00000001:3.0:1782489572.012217:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303816730 00000100:00000001:2.0:1782489572.012217:0:2419:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012219:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303820480 00000100:00000001:3.0:1782489572.012221:0:2418:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.012223:0:2418:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012224:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012225:0:2418:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000001:2.0:1782489572.012226:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489572.012229:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012233:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489572.012234:0:2418:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1782489572.012235:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1782489572.012236:0:2418:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012237:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000001:3.0:1782489572.012238:0:2418:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:2.0:1782489572.012238:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1782489572.012239:0:2419:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 02000000:00000001:3.0:1782489572.012240:0:2418:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1782489572.012241:0:2418:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1782489572.012243:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489572.012244:0:2418:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5b6bfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685824:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489572.012247:0:2418:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:2.0:1782489572.012249:0:2419:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1782489572.012250:0:2418:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:2.0:1782489572.012250:0:2419:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.012251:0:2418:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1782489572.012252:0:2419:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1782489572.012253:0:2418:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:2.0:1782489572.012254:0:2419:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000010:3.0:1782489572.012256:0:2418:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fa6000. 02000000:00000001:2.0:1782489572.012256:0:2419:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.012258:0:2418:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1782489572.012259:0:2419:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9ce5b6bf9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2419:1869066440685696:192.168.202.139@tcp:400:kworker.0 00000400:00000200:3.0:1782489572.012261:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:2.0:1782489572.012262:0:2419:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:3.0:1782489572.012264:0:2418:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce5a62d9aa8. 02000000:00000001:2.0:1782489572.012264:0:2419:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1782489572.012265:0:2419:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1782489572.012266:0:2419:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1782489572.012267:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09a7f8. 02000000:00000010:2.0:1782489572.012269:0:2419:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9ce584fada00. 00000100:00000200:3.0:1782489572.012270:0:2418:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685824, portal 4 02000000:00000001:2.0:1782489572.012270:0:2419:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012272:0:2418:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1782489572.012273:0:2418:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109160298112 : -108964549253504 : ffff9ce5b6bfa680) 00000400:00000200:2.0:1782489572.012273:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000010:2.0:1782489572.012275:0:2419:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9ce599649058. 00000400:00000010:2.0:1782489572.012278:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce588d97aa0. 00000100:00000040:3.0:1782489572.012279:0:2418:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1782489572.012281:0:2419:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1869066440685696, portal 10 00000100:00000001:2.0:1782489572.012283:0:2419:0:(client.c:3274:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1782489572.012284:0:2419:0:(client.c:3276:ptlrpc_request_addref()) Process leaving (rc=18446635109160294528 : -108964549257088 : ffff9ce5b6bf9880) 00000100:00000001:3.0:1782489572.012286:0:2418:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1782489572.012288:0:2418:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000100:00000040:2.0:1782489572.012289:0:2419:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:3.0:1782489572.012290:0:2418:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce5ba09abb0. 00000100:00000001:2.0:1782489572.012295:0:2419:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000200:3.0:1782489572.012296:0:2418:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1869066440685824, offset 0 00000100:00000040:2.0:1782489572.012297:0:2419:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.139@tcp 00000400:00000010:2.0:1782489572.012299:0:2419:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9ce588d974c8. 00000400:00000200:3.0:1782489572.012300:0:2418:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000100:00000200:2.0:1782489572.012302:0:2419:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1869066440685696, offset 0 00000400:00000200:3.0:1782489572.012304:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.012306:0:2419:0:(lib-move.c:5069:LNetPut()) LNetPut -> 12345-192.168.202.139@tcp 00000400:00000200:3.0:1782489572.012307:0:2418:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000400:00000200:3.0:1782489572.012309:0:2418:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1782489572.012311:0:2418:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000400:00000200:3.0:1782489572.012313:0:2418:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1782489572.012316:0:2418:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000400:00000200:3.0:1782489572.012318:0:2418:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536816:536816:254:4294967295] 192.168.202.139@tcp LPNI seq info [536783:536783:6:4294967295] 00000400:00000200:3.0:1782489572.012322:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489572.012328:0:2418:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000800:00000200:3.0:1782489572.012332:0:2418:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000400:00000200:2.0:1782489572.012332:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000800:00000010:3.0:1782489572.012335:0:2418:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8ba9300. 00000400:00000200:2.0:1782489572.012336:0:2419:0:(lib-move.c:2720:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.139@tcp local destination 00000800:00000200:3.0:1782489572.012338:0:2418:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000400:00000200:2.0:1782489572.012338:0:2419:0:(lib-move.c:2350:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1782489572.012340:0:2419:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.202.39@tcp 00000800:00000200:3.0:1782489572.012343:0:2418:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000400:00000200:2.0:1782489572.012343:0:2419:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.202.39@tcp lpni_is_preferred = 0 00000800:00000200:3.0:1782489572.012345:0:2418:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8ba9300 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:2.0:1782489572.012345:0:2419:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.139@tcp 00000100:00000001:3.0:1782489572.012348:0:2418:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1782489572.012348:0:2419:0:(lib-move.c:1652:lnet_handle_send()) 192.168.202.39@tcp NI seq info: [536817:536817:253:4294967295] 192.168.202.139@tcp LPNI seq info [536784:536784:5:4294967295] 00000100:00000040:3.0:1782489572.012351:0:2418:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: using a timeout of 91940 usecs (92 jiffies) 00000400:00000200:2.0:1782489572.012351:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:3.0:1782489572.012354:0:2418:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489572.012356:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303816796 00000400:00000200:2.0:1782489572.012357:0:2419:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp:) -> 192.168.202.139@tcp(192.168.202.139@tcp:192.168.202.139@tcp) : PUT try# 0 00000100:00000001:3.0:1782489572.012358:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303820480 00000100:00000001:3.0:1782489572.012360:0:2418:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303816796 00000800:00000200:2.0:1782489572.012361:0:2419:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.139@tcp 00000100:00000001:3.0:1782489572.012362:0:2418:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303820480 00000100:00000001:3.0:1782489572.012364:0:2418:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000010:2.0:1782489572.012364:0:2419:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9ce5b8602200. 00000100:00000001:3.0:1782489572.012366:0:2418:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1782489572.012367:0:2419:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9ce587df3240] -> 12345-192.168.202.139@tcp (5) 00000100:00000001:3.0:1782489572.012368:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012370:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000800:00000200:2.0:1782489572.012371:0:2419:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.139@tcp ip 192.168.202.139:988 00000100:00000001:3.0:1782489572.012373:0:2418:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000800:00000200:2.0:1782489572.012373:0:2419:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9ce5b8602200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1782489572.012375:0:2418:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1782489572.012376:0:2419:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.012377:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000040:2.0:1782489572.012378:0:2419:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: using a timeout of 25113 usecs (26 jiffies) 00000400:00000200:2.0:1782489572.012381:0:2419:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000100:00000001:3.0:1782489572.012382:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489572.012383:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012383:0:2419:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 2 is 4303816730 00000100:00000001:3.0:1782489572.012386:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012386:0:2419:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 2, keep current deadline 4303820480 00000100:00000001:2.0:1782489572.012388:0:2419:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS new deadline estimation for cpu 3 is 4303816730 00000100:00000001:2.0:1782489572.012389:0:2419:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9ce5830d8800: PM QoS request active for cpu 3, keep current deadline 4303820480 00000100:00000001:2.0:1782489572.012391:0:2419:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1782489572.012393:0:2419:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012395:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012397:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1782489572.012400:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1782489572.012401:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1782489572.012403:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1782489572.012404:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1782489572.012405:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012407:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:2.0:1782489572.012479:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489572.012484:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8602c00. 00000400:00000200:2.0:1782489572.012490:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.012496:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489572.012501:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce588d97dd0 00000400:00000010:2.0:1782489572.012504:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce588d97dd0. 00000100:00000001:2.0:1782489572.012508:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489572.012513:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012521:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489572.012524:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012530:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012531:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489572.012542:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489572.012544:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9500. 00000400:00000200:2.0:1782489572.012547:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.012549:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489572.012552:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09ad48 00000400:00000010:2.0:1782489572.012553:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09ad48. 00000100:00000001:2.0:1782489572.012555:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489572.012558:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012564:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489572.012567:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012574:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012575:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489572.012582:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489572.012584:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8ba9300. 00000400:00000200:2.0:1782489572.012586:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.012589:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489572.012591:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09abb0 00000400:00000010:2.0:1782489572.012598:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09abb0. 00000100:00000001:2.0:1782489572.012601:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489572.012604:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012611:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489572.012613:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012619:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012620:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:2.0:1782489572.012685:0:2410:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:2.0:1782489572.012688:0:2410:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9ce5b8602200. 00000400:00000200:2.0:1782489572.012691:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.012694:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000400:00000200:2.0:1782489572.012696:0:2410:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce588d974c8 00000400:00000010:2.0:1782489572.012698:0:2410:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce588d974c8. 00000100:00000001:2.0:1782489572.012700:0:2410:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:2.0:1782489572.012703:0:2410:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012710:0:2410:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1782489572.012713:0:2410:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.012719:0:2410:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1782489572.012720:0:2410:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1782489572.018319:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.018333:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489572.018337:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.018340:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.018348:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.018365:0:2411:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489572.018374:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.018379:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.018383:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.018387:0:2411:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 4 MB=0x6a3e7f1423900 00000400:00000200:2.0:1782489572.018403:0:2411:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.018409:0:2411:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9cb5 [1] + 224 00000800:00000001:2.0:1782489572.018417:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.018424:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.018427:0:2411:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.018449:0:2411:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489572.018457:0:2411:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489572.018466:0:2411:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489572.018478:0:2411:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.018527:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.018531:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1782489572.018593:0:2411:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489572.018603:0:2411:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.018607:0:2411:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:3.0:1782489572.018653:0:2418:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489572.018656:0:2418:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489572.018659:0:2418:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9aa8. 00000400:00000200:3.0:1782489572.018663:0:2418:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09a7f8 00000400:00000010:3.0:1782489572.018666:0:2418:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09a7f8. 00000100:00000001:3.0:1782489572.018669:0:2418:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489572.018674:0:2418:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489572.018728:0:2418:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.018735:0:2418:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489572.018737:0:2418:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.018739:0:2418:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489572.018742:0:2418:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489572.018745:0:2418:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489572.018747:0:2418:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.018750:0:2418:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489572.018760:0:2418:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5b6bc3c60 time=53 v=5 (5 5 5 5) 00000100:00001000:3.0:1782489572.018764:0:2418:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3c18 time=130 v=5 (1 1 0 0) 00000100:00000001:3.0:1782489572.018767:0:2418:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489572.018769:0:2418:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.018771:0:2418:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1782489572.018775:0:2418:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.018779:0:2418:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489572.018782:0:2418:0:(client.c:3107:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9ce5830d8800: skip recheck: last_committed 85899368364 00000100:00000001:3.0:1782489572.018784:0:2418:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489572.018786:0:2418:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.018790:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.018797:0:2418:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.018809:0:2418:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489572.018812:0:2418:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489572.018816:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489572.018823:0:2418:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5b6bfa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685824:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489572.018827:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.018829:0:2418:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489572.018833:0:2418:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5b6bfa680 x1869066440685824/t0(0) o400->lustre-OST0000-osc-ffff9ce5830d8800@192.168.202.139@tcp:28/4 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.018840:0:2418:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489572.018842:0:2418:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489572.018845:0:2418:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fa6000. 02000000:00000001:3.0:1782489572.018849:0:2418:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489572.018851:0:2418:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489572.018853:0:2418:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3800 refcount=3 obd=lustre-OST0000-osc-ffff9ce5830d8800 00000020:00000001:3.0:1782489572.018855:0:2418:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489572.018857:0:2418:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8ba9700. 02000000:00000001:3.0:1782489572.018861:0:2418:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489572.018863:0:2418:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489572.018865:0:2418:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5b6bfa680. 00000100:00000001:3.0:1782489572.018868:0:2418:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489572.018870:0:2418:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.018872:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.018874:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.018875:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1782489572.018877:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.018878:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1782489572.026543:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026560:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489572.026566:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026568:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026577:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026584:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489572.026593:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026597:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026603:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026607:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 25 MB=0x6a3e7f1423780 00000400:00000200:2.0:1782489572.026612:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026618:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c95 [1] + 224 00000800:00000001:2.0:1782489572.026624:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026628:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026631:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.026635:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489572.026642:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489572.026650:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489572.026674:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.026691:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489572.026698:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026701:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489572.026703:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026705:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026707:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026711:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489572.026715:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026718:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026720:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026723:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423800 00000400:00000200:2.0:1782489572.026727:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026731:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9c9d [1] + 224 00000800:00000001:2.0:1782489572.026735:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026738:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026740:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.026743:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489572.026746:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489572.026750:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489572.026758:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.026770:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489572.026773:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026775:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1782489572.026776:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026778:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026780:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026787:0:2410:0:(lib-move.c:4560:lnet_parse()) TRACE: 192.168.202.39@tcp(192.168.202.39@tcp) <- 192.168.202.139@tcp : PUT - for me 00000400:00000200:2.0:1782489572.026791:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026793:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026796:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026799:0:2410:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.202.139@tcp of length 224 into portal 10 MB=0x6a3e7f1423880 00000400:00000200:2.0:1782489572.026802:0:2410:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.202.139@tcp to cpt 1 00000400:00000200:2.0:1782489572.026805:0:2410:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.139@tcp of length 224/224 into md 0x4e9cbd [1] + 224 00000800:00000001:2.0:1782489572.026808:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026813:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1782489572.026815:0:2410:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1782489572.026818:0:2410:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.202.39@tcp->192.168.202.139@tcp: PUT: OK 00000100:00000001:2.0:1782489572.026821:0:2410:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1782489572.026825:0:2410:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1782489572.026833:0:2410:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1782489572.026840:0:2410:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1782489572.026842:0:2410:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1782489572.026844:0:2410:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027012:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.027016:0:2419:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1782489572.027022:0:2419:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489572.027024:0:2419:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489572.027027:0:2419:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce599649058. 00000400:00000200:3.0:1782489572.027031:0:2419:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce588d97aa0 00000400:00000010:3.0:1782489572.027033:0:2419:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce588d97aa0. 00000100:00000001:3.0:1782489572.027036:0:2419:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489572.027046:0:2419:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489572.027055:0:2419:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027061:0:2419:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489572.027063:0:2419:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027066:0:2419:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489572.027068:0:2419:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489572.027071:0:2419:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489572.027073:0:2419:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.027076:0:2419:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489572.027086:0:2419:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc1460 time=53 v=60 (60 60 60 60) 00000100:00001000:3.0:1782489572.027090:0:2419:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc1418 time=77 v=5 (1 1 1 1) 00000100:00000001:3.0:1782489572.027094:0:2419:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489572.027095:0:2419:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.027098:0:2419:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1782489572.027102:0:2419:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1782489572.027110:0:2419:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027113:0:2419:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489572.027115:0:2419:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9ce5830d8800: skip recheck: last_committed 21474836540 00000100:00000001:3.0:1782489572.027117:0:2419:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489572.027118:0:2419:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.027122:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027152:0:2419:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.027155:0:2419:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489572.027158:0:2419:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489572.027162:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489572.027171:0:2419:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5b6bf9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:bf41b9ca-a66c-4f13-9d79-fff1432569da:2419:1869066440685696:192.168.202.139@tcp:400:kworker.0 00000400:00000001:3.0:1782489572.027181:0:2419:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489572.027183:0:2419:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489572.027185:0:2419:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce599649d10. 00000400:00000200:3.0:1782489572.027187:0:2419:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce588d97088 00000400:00000010:3.0:1782489572.027189:0:2419:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce588d97088. 00000100:00000001:3.0:1782489572.027191:0:2419:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489572.027195:0:2419:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489572.027203:0:2419:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027217:0:2419:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489572.027218:0:2419:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027220:0:2419:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489572.027222:0:2419:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489572.027223:0:2419:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489572.027224:0:2419:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.027226:0:2419:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489572.027229:0:2419:0:(import.c:2151:obd_at_measure()) add 5 to ffff9ce5834b1c60 time=53 v=5 (5 5 5 5) 00000100:00001000:3.0:1782489572.027232:0:2419:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5834b1c18 time=89 v=5 (1 1 1 1) 00000100:00000001:3.0:1782489572.027234:0:2419:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489572.027235:0:2419:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.027237:0:2419:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1782489572.027239:0:2419:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027240:0:2419:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.027244:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027249:0:2419:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.027251:0:2419:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489572.027252:0:2419:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489572.027255:0:2419:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489572.027262:0:2419:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5b6bfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:dd22dfa7-56fe-4502-9376-64e8b3112148:2419:1869066440685440:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489572.027270:0:2419:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027273:0:2419:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489572.027276:0:2419:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5b6bf9880 x1869066440685696/t0(0) o400->lustre-MDT0001-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027284:0:2419:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489572.027287:0:2419:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489572.027290:0:2419:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fada00. 02000000:00000001:3.0:1782489572.027293:0:2419:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489572.027296:0:2419:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489572.027299:0:2419:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc1000 refcount=3 obd=lustre-MDT0001-mdc-ffff9ce5830d8800 00000020:00000001:3.0:1782489572.027302:0:2419:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489572.027305:0:2419:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8ba9100. 02000000:00000001:3.0:1782489572.027308:0:2419:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489572.027310:0:2419:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489572.027313:0:2419:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5b6bf9880. 00000100:00000001:3.0:1782489572.027317:0:2419:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489572.027318:0:2419:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027320:0:2419:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489572.027322:0:2419:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5b6bfbb80 x1869066440685440/t0(0) o400->MGC192.168.202.139@tcp@192.168.202.139@tcp:26/25 lens 224/224 e 0 to 0 dl 1782489588 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027328:0:2419:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489572.027329:0:2419:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489572.027330:0:2419:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fac800. 02000000:00000001:3.0:1782489572.027332:0:2419:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489572.027333:0:2419:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489572.027334:0:2419:0:(genops.c:1244:class_import_put()) import ffff9ce5834b1800 refcount=3 obd=MGC192.168.202.139@tcp 00000020:00000001:3.0:1782489572.027336:0:2419:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489572.027337:0:2419:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8ba9900. 02000000:00000001:3.0:1782489572.027338:0:2419:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489572.027339:0:2419:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489572.027340:0:2419:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5b6bfbb80. 00000100:00000001:3.0:1782489572.027342:0:2419:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489572.027343:0:2419:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027346:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027352:0:2419:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489572.027357:0:2419:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027360:0:2419:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.027362:0:2419:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027371:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.027372:0:2418:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1782489572.027375:0:2418:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1782489572.027377:0:2418:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1782489572.027378:0:2418:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9ce5a62d9a50. 00000400:00000200:3.0:1782489572.027381:0:2418:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9ce5ba09acc0 00000400:00000010:3.0:1782489572.027383:0:2418:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff9ce5ba09acc0. 00000100:00000001:3.0:1782489572.027385:0:2418:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1782489572.027390:0:2418:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1782489572.027398:0:2418:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027403:0:2418:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1782489572.027405:0:2418:0:(client.c:2987:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027407:0:2418:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:3.0:1782489572.027408:0:2418:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1782489572.027409:0:2418:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1782489572.027411:0:2418:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1782489572.027412:0:2418:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1782489572.027417:0:2418:0:(import.c:2151:obd_at_measure()) add 60 to ffff9ce5b6bc3460 time=53 v=60 (60 60 60 60) 00000100:00001000:3.0:1782489572.027420:0:2418:0:(import.c:2151:obd_at_measure()) add 1 to ffff9ce5b6bc3418 time=89 v=5 (1 1 1 1) 00000100:00000001:3.0:1782489572.027422:0:2418:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1782489572.027423:0:2418:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.027425:0:2418:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1782489572.027427:0:2418:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027478:0:2418:0:(client.c:3095:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1782489572.027480:0:2418:0:(client.c:3107:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9ce5830d8800: skip recheck: last_committed 30064771619 00000100:00000001:3.0:1782489572.027482:0:2418:0:(client.c:3108:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1782489572.027483:0:2418:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1782489572.027487:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027493:0:2418:0:(client.c:2340:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1782489572.027500:0:2418:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1782489572.027501:0:2418:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1782489572.027505:0:2418:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1782489572.027512:0:2418:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9ce5b6bfb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:bf41b9ca-a66c-4f13-9d79-fff1432569da:2418:1869066440685568:192.168.202.139@tcp:400:kworker.0 00000100:00000001:3.0:1782489572.027515:0:2418:0:(client.c:2451:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027517:0:2418:0:(client.c:2853:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1782489572.027520:0:2418:0:(client.c:2861:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9ce5b6bfb800 x1869066440685568/t0(0) o400->lustre-MDT0000-mdc-ffff9ce5830d8800@192.168.202.139@tcp:12/10 lens 224/224 e 0 to 0 dl 1782489657 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1782489572.027526:0:2418:0:(client.c:2779:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1782489572.027528:0:2418:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1782489572.027529:0:2418:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9ce584fa6200. 02000000:00000001:3.0:1782489572.027531:0:2418:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1782489572.027533:0:2418:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1782489572.027534:0:2418:0:(genops.c:1244:class_import_put()) import ffff9ce5b6bc3000 refcount=8 obd=lustre-MDT0000-mdc-ffff9ce5830d8800 00000020:00000001:3.0:1782489572.027535:0:2418:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1782489572.027537:0:2418:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9ce5b8ba9400. 02000000:00000001:3.0:1782489572.027539:0:2418:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1782489572.027540:0:2418:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1782489572.027541:0:2418:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9ce5b6bfb800. 00000100:00000001:3.0:1782489572.027543:0:2418:0:(client.c:2835:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1782489572.027545:0:2418:0:(client.c:2894:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027546:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1782489572.027549:0:2418:0:(client.c:2625:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1782489572.027550:0:2418:0:(client.c:2653:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1782489572.027552:0:2418:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1782489572.027553:0:2418:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:2.0:1782489572.229328:0:286660:0:(llite_lib.c:4108:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.523975:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489572.523981:0:213139:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:3.0:1782489572.523984:0:213139:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:3.0:1782489572.523987:0:213139:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.523991:0:213139:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:3.0:1782489572.523993:0:213139:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.523995:0:213139:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.524014:0:213139:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489572.524018:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489572.524020:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1782489572.524290:0:238718:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1782489572.524294:0:238718:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1782489572.524297:0:238718:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000001:1.0:1782489572.617540:0:286697:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:1.0:1782489572.617547:0:286697:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_816: @@@@@@ FAIL: client: import is not in IDLE state after 50 00000001:00000001:1.0:1782489572.632057:0:286697:0:(debug.c:656:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:3.0:1782489573.549236:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489573.549243:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1782489573.549249:0:213139:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:3.0:1782489573.549250:0:213139:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1782489573.549757:0:254078:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1782489573.549766:0:254078:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1782489573.549770:0:254078:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1782489574.094064:0:286797:0:(llite_lib.c:4108:ll_show_options()) Process leaving (rc=0 : 0 : 0) Debug log: 2669 lines, 2669 kept, 0 dropped, 0 bad.