00010000:00000001:2.0F:1788536079.717759:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536079.717775:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536079.717782:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536079.717783:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0F:1788536079.717831:0:248051:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536079.717839:0:248051:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536079.717842:0:248051:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788536080.613689:0:265135:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5a20 (cpu 2) has reached its deadline 4305376528, at 4305376576 00000100:00000010:2.0:1788536080.613707:0:265135:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b2472f58960. 00000100:00000040:3.0F:1788536080.613781:0:260093:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5b30 (cpu 3) has reached its deadline 4305376528, at 4305376576 00000100:00000010:3.0:1788536080.613796:0:260093:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b2472f587e0. 00010000:00000001:2.0:1788536080.741668:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536080.741675:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536080.741680:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536080.741681:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0F:1788536080.743299:0:252172:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536080.743320:0:252172:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536080.743325:0:252172:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000200:2.0:1788536081.702933:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b24523f16c0] -> 12345-192.168.206.23@tcp (6) 00000800:00000200:2.0:1788536081.702942:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.23@tcp ip 192.168.206.23:988 00000800:00000200:2.0:1788536081.702947:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b244334a600 type 192, nob 24 niov 1 nkiov 0 00000800:00000200:3.0:1788536081.703315:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000001:3.0:1788536081.703321:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.703349:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536081.703353:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.703355:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788536081.767658:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536081.767664:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536081.767670:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536081.767671:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788536081.767805:0:236365:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788536081.767809:0:236365:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788536081.767811:0:236365:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788536081.834059:0:233563:0:(pinger.c:229:ptlrpc_pinger_process_import()) f1e73104-a124-445d-9e99-edaf7c361d90->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788536081.834069:0:233563:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788536081.834076:0:233563:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2448436d80. 00000020:00000040:2.0:1788536081.834081:0:233563:0:(genops.c:1210:class_import_get()) import ffff9b244a7c6000 refcount=4 obd=MGC192.168.206.123@tcp 00000100:00000001:2.0:1788536081.834085:0:233563:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788536081.834087:0:233563:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788536081.834092:0:233563:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788536081.834100:0:233563:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b24722e5400. 00000100:00000001:2.0:1788536081.834106:0:233563:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.834117:0:233563:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging f1e73104-a124-445d-9e99-edaf7c361d90->MGS req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788536081.834127:0:233563:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788536081.834131:0:233563:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.834137:0:233563:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006eea3495] to pc [ptlrpcd_01_00+0] req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.834202:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.834206:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.834209:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536081.834215:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.834222:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536081.834224:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.834228:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536081.834230:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536081.834233:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.834236:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2448436d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f1e73104-a124-445d-9e99-edaf7c361d90:2405:1875404964675712:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536081.834240:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536081.834256:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536081.834258:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.834261:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536081.834265:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2450f5e800. 02000000:00000001:3.0:1788536081.834267:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536081.834274:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536081.834278:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b420. 00000400:00000010:3.0:1788536081.834285:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1b28. 00000100:00000200:3.0:1788536081.834291:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964675712, portal 25 00000100:00000001:3.0:1788536081.834293:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536081.834295:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633178866347392 : -110894843204224 : ffff9b2448436d80) 00000100:00000040:3.0:1788536081.834300:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.834319:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536081.834323:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536081.834325:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd12a8. 00000100:00000200:3.0:1788536081.834329:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875404964675712, offset 0 00000400:00000200:3.0:1788536081.834335:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536081.834343:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.834349:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536081.834354:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536081.834358:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536081.834362:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536081.834365:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536081.834368:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539855:539855:256:4294967295] 192.168.206.123@tcp LPNI seq info [539806:539806:8:4294967295] 00000400:00000200:3.0:1788536081.834373:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.834381:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536081.834386:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536081.834392:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34d500. 00000800:00000200:3.0:1788536081.834398:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536081.834405:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536081.834409:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34d500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536081.834423:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536081.834427:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536081.834429:0:2405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 2 is 4305381651 00000100:00000010:3.0:1788536081.834433:0:2405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b24480ed9c0. 00000100:00000001:3.0:1788536081.834439:0:2405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788536081.834445:0:2405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 3 is 4305381651 00000100:00000010:3.0:1788536081.834448:0:2405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b24480ed5a0. 00000100:00000001:3.0:1788536081.834450:0:2405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788536081.834453:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.834456:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.834464:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.834466:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.834489:0:233563:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.834496:0:233563:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788536081.834499:0:233563:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788536081.834502:0:233563:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2448434700. 00000020:00000040:2.0:1788536081.834504:0:233563:0:(genops.c:1210:class_import_get()) import ffff9b2472549000 refcount=6 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000100:00000001:2.0:1788536081.834507:0:233563:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788536081.834508:0:233563:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788536081.834510:0:233563:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788536081.834514:0:233563:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b24722e5000. 00000100:00000001:2.0:1788536081.834517:0:233563:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.834522:0:233563:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788536081.834530:0:233563:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788536081.834532:0:233563:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.834536:0:233563:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000066e3f5c0] to pc [ptlrpcd_01_01+1] req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.835700:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536081.835704:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788536081.835708:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.835710:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.835712:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.835714:0:2405:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788536081.835717:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.835719:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536081.835721:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.835722:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.835723:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.835724:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536081.835732:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.835741:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536081.835742:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.835745:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536081.835747:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536081.835750:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.835753:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2448434700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964675840:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536081.835757:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536081.835760:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536081.835762:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.835764:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536081.835768:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2450f5e600. 02000000:00000001:3.0:1788536081.835770:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536081.835775:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536081.835778:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b790. 00000400:00000010:3.0:1788536081.835784:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1bb0. 00000100:00000200:3.0:1788536081.835788:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964675840, portal 10 00000100:00000001:3.0:1788536081.835790:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536081.835792:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633178866337536 : -110894843214080 : ffff9b2448434700) 00000100:00000040:3.0:1788536081.835799:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.835812:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536081.835814:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536081.835817:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1770. 00000100:00000200:3.0:1788536081.835820:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964675840, offset 0 00000400:00000200:3.0:1788536081.835826:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536081.835832:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.835838:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536081.835842:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536081.835845:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536081.835849:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536081.835851:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536081.835855:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539856:539856:255:4294967295] 192.168.206.123@tcp LPNI seq info [539807:539807:7:4294967295] 00000400:00000200:3.0:1788536081.835860:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.835868:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536081.836677:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536081.836684:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34d700. 00000800:00000200:3.0:1788536081.836689:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536081.836695:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536081.836699:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34d700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536081.836703:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536081.836707:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: using a timeout of 19382 usecs (20 jiffies) 00000400:00000200:3.0:1788536081.836711:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536081.836714:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305377819 00000100:00000001:3.0:1788536081.836718:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.836727:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305377819 00000100:00000001:3.0:1788536081.836730:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.836732:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.836735:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.836760:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.836764:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.836780:0:2406:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.836782:0:2406:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788536081.836832:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536081.836836:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34d500. 00000400:00000200:3.0:1788536081.836842:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.836847:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536081.836865:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd12a8 00000400:00000010:3.0:1788536081.836867:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd12a8. 00000100:00000001:3.0:1788536081.836872:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536081.836878:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.837001:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.837096:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.837104:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.837105:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536081.837175:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536081.837186:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34d700. 00000400:00000200:3.0:1788536081.837190:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.837194:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536081.837197:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1770 00000400:00000010:3.0:1788536081.837199:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1770. 00000100:00000001:3.0:1788536081.837201:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536081.837222:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.837231:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.837234:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.837249:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.837251:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788536081.838086:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.838095:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.838098:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838101:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.838150:0:233563:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838154:0:233563:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788536081.838158:0:233563:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788536081.838163:0:233563:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2448434380. 00000020:00000040:2.0:1788536081.838167:0:233563:0:(genops.c:1210:class_import_get()) import ffff9b247254c000 refcount=4 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000100:00000001:2.0:1788536081.838171:0:233563:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788536081.838174:0:233563:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788536081.838178:0:233563:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788536081.838183:0:233563:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b24722e5200. 00000100:00000001:2.0:1788536081.838192:0:233563:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838200:0:233563:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788536081.838209:0:233563:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788536081.838212:0:233563:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838217:0:233563:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000000907b6f] to pc [ptlrpcd_01_00+0] req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838276:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.838279:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.838280:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536081.838286:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838293:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536081.838295:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.838298:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536081.838301:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536081.838303:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.838307:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2448434380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964675968:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536081.838310:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536081.838315:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536081.838317:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.838319:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536081.838323:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2450f5f600. 02000000:00000001:3.0:1788536081.838325:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536081.838330:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536081.838334:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15bdc0. 00000400:00000010:3.0:1788536081.838339:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1770. 00000100:00000200:3.0:1788536081.838344:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964675968, portal 10 00000100:00000001:3.0:1788536081.838346:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536081.838348:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633178866336640 : -110894843214976 : ffff9b2448434380) 00000100:00000040:3.0:1788536081.838355:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838363:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536081.838365:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536081.838368:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd12a8. 00000100:00000200:3.0:1788536081.838370:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964675968, offset 0 00000400:00000200:3.0:1788536081.838376:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536081.838384:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.838396:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536081.838401:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536081.838405:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536081.838408:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536081.838411:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536081.838414:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539857:539857:256:4294967295] 192.168.206.123@tcp LPNI seq info [539808:539808:8:4294967295] 00000400:00000200:3.0:1788536081.838419:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.838426:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536081.838431:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536081.838436:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34d700. 00000800:00000200:3.0:1788536081.838442:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536081.838447:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536081.838450:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34d700 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536081.838461:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536081.838464:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: using a timeout of 20280 usecs (21 jiffies) 00000400:00000200:3.0:1788536081.838467:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536081.838469:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305377821 00000100:00000001:3.0:1788536081.838472:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.838474:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305377821 00000100:00000001:3.0:1788536081.838477:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.838479:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.838482:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838485:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838487:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.838512:0:233563:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838515:0:233563:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:2.0:1788536081.838519:0:233563:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:2.0:1788536081.838522:0:233563:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2448434e00. 00000020:00000040:2.0:1788536081.838525:0:233563:0:(genops.c:1210:class_import_get()) import ffff9b247254e800 refcount=4 obd=lustre-OST0000-osc-ffff9b244134e800 00000100:00000001:2.0:1788536081.838528:0:233563:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:2.0:1788536081.838529:0:233563:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:2.0:1788536081.838532:0:233563:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:2.0:1788536081.838535:0:233563:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b24722e5b00. 00000100:00000001:2.0:1788536081.838539:0:233563:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838544:0:233563:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:2.0:1788536081.838552:0:233563:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:2.0:1788536081.838554:0:233563:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.838558:0:233563:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b9c3d39b] to pc [ptlrpcd_01_01+1] req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838716:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536081.838718:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788536081.838721:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.838722:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.838724:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.838727:0:2405:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:3.0:1788536081.838728:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.838737:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536081.838739:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.838740:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.838742:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.838743:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536081.838748:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838755:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536081.838757:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.838760:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536081.838761:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536081.838764:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536081.838767:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2448434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676096:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536081.838770:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536081.838773:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536081.838774:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.838776:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536081.838780:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b24521da800. 02000000:00000001:3.0:1788536081.838781:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536081.838785:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536081.838788:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b688. 00000400:00000010:3.0:1788536081.838792:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1dd0. 00000100:00000200:3.0:1788536081.838795:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676096, portal 4 00000100:00000001:3.0:1788536081.838797:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536081.838798:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633178866339328 : -110894843212288 : ffff9b2448434e00) 00000100:00000040:3.0:1788536081.838805:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838812:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536081.838815:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536081.838817:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd13b8. 00000100:00000200:3.0:1788536081.838819:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875404964676096, offset 0 00000400:00000200:3.0:1788536081.838824:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536081.838828:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.838836:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536081.838839:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536081.838842:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536081.838845:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536081.838847:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536081.838851:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539858:539858:255:4294967295] 192.168.206.123@tcp LPNI seq info [539809:539809:7:4294967295] 00000400:00000200:3.0:1788536081.838855:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.838862:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536081.838866:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536081.838870:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34d500. 00000800:00000200:3.0:1788536081.838874:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536081.838878:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536081.838881:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34d500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536081.838884:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536081.838887:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: using a timeout of 116181 usecs (117 jiffies) 00000400:00000200:3.0:1788536081.838891:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536081.838892:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305377918 00000100:00000001:3.0:1788536081.838894:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.838897:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305377918 00000100:00000001:3.0:1788536081.838899:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305381651 00000100:00000001:3.0:1788536081.838901:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.838903:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838905:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838907:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.838917:0:2406:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.838919:0:2406:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788536081.838943:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536081.838946:0:2398:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34d700. 00000400:00000200:3.0:1788536081.838951:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.838955:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536081.838966:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd12a8 00000400:00000010:3.0:1788536081.838968:0:2398:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd12a8. 00000100:00000001:3.0:1788536081.838982:0:2398:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536081.838987:0:2398:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.838995:0:2398:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.838998:0:2398:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.839004:0:2398:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.839006:0:2398:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536081.839276:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536081.839280:0:2398:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34d500. 00000400:00000200:3.0:1788536081.839283:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.839287:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536081.839290:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd13b8 00000400:00000010:3.0:1788536081.839292:0:2398:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd13b8. 00000100:00000001:3.0:1788536081.839294:0:2398:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536081.839298:0:2398:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.839306:0:2398:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.839309:0:2398:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.839315:0:2398:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.839316:0:2398:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:3.0:1788536081.839434:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.839437:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536081.839439:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.839441:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.839468:0:233563:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.839471:0:233563:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:2.0:1788536081.839475:0:233563:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:2.0:1788536081.839480:0:233563:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (10715) 00000800:00000001:3.0:1788536081.842172:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842188:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536081.842192:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842194:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.842202:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842209:0:2397:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536081.842223:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842227:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842232:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842236:0:2397:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 25 MB=0x6a9abbebac080 00000400:00000200:3.0:1788536081.842302:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842310:0:2397:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508dcd [1] + 224 00000800:00000001:3.0:1788536081.842318:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842324:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.842327:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.842332:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536081.842339:0:2397:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.842347:0:2397:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536081.842359:0:2397:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.842383:0:2397:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536081.842391:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842395:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536081.842397:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842400:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.842403:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842406:0:2397:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536081.842411:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842413:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842416:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842419:0:2397:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac100 00000400:00000200:3.0:1788536081.842422:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.842427:0:2397:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508ddd [1] + 224 00000800:00000001:3.0:1788536081.842431:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842436:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.842438:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.842447:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536081.842450:0:2397:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.842455:0:2397:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536081.842464:0:2397:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.842471:0:2397:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536081.842476:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.842479:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.842552:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788536081.842557:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:2.0:1788536081.842679:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788536081.842682:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788536081.842685:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b790. 00000400:00000200:2.0:1788536081.842690:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1bb0 00000400:00000010:2.0:1788536081.842693:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1bb0. 00000100:00000001:2.0:1788536081.842697:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788536081.842701:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788536081.842725:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842731:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788536081.842733:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.842736:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788536081.842739:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788536081.842741:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788536081.842743:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788536081.842746:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788536081.842754:0:2405:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b2472549460 time=108 v=60 (60 61 61 62) 00000100:00001000:2.0:1788536081.842759:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b2472549418 time=89 v=5 (1 2 0 0) 00000100:00000001:2.0:1788536081.842761:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788536081.842763:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536081.842766:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788536081.842769:0:2405:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.842772:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1788536081.842775:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9b244134e800: skip recheck: last_committed 30064771462 00000100:00000001:2.0:1788536081.842776:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1788536081.842778:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.842783:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842791:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788536081.842794:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788536081.842796:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788536081.842801:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788536081.842809:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2448434700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964675840:192.168.206.123@tcp:400:kworker.0 00000400:00000001:2.0:1788536081.842815:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1788536081.842817:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1788536081.842818:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b420. 00000400:00000200:2.0:1788536081.842826:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1b28 00000400:00000010:2.0:1788536081.842828:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1b28. 00000100:00000001:2.0:1788536081.842830:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788536081.842834:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:2.0:1788536081.842843:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842850:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1788536081.842852:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.842853:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:2.0:1788536081.842855:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:2.0:1788536081.842856:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:2.0:1788536081.842858:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1788536081.842860:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1788536081.842862:0:2405:0:(import.c:2149:obd_at_measure()) add 5 to ffff9b244a7c6460 time=108 v=5 (5 5 5 5) 00000100:00001000:2.0:1788536081.842865:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b244a7c6418 time=89 v=5 (1 1 0 0) 00000100:00000001:2.0:1788536081.842867:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1788536081.842869:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536081.842870:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:2.0:1788536081.842872:0:2405:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.842874:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1788536081.842878:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842885:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1788536081.842887:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1788536081.842888:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1788536081.842892:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1788536081.842899:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2448436d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f1e73104-a124-445d-9e99-edaf7c361d90:2405:1875404964675712:192.168.206.123@tcp:400:kworker.0 00000100:00000001:2.0:1788536081.842903:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.842905:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788536081.842908:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2448434700 x1875404964675840/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842921:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788536081.842923:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788536081.842926:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2450f5e600. 02000000:00000001:2.0:1788536081.842930:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788536081.842932:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788536081.842934:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b2472549000 refcount=5 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000020:00000001:2.0:1788536081.842936:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788536081.842938:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b24722e5000. 02000000:00000001:2.0:1788536081.842941:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788536081.842943:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788536081.842945:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2448434700. 00000100:00000001:2.0:1788536081.842948:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788536081.842949:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.842951:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1788536081.842954:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2448436d80 x1875404964675712/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536097 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536081.842960:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1788536081.842961:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1788536081.842963:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2450f5e800. 02000000:00000001:2.0:1788536081.842965:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1788536081.842966:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:2.0:1788536081.842967:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b244a7c6000 refcount=3 obd=MGC192.168.206.123@tcp 00000020:00000001:2.0:1788536081.842969:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:2.0:1788536081.842971:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b24722e5400. 02000000:00000001:2.0:1788536081.843086:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1788536081.843088:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1788536081.843089:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2448436d80. 00000100:00000001:2.0:1788536081.843092:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1788536081.843094:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1788536081.843097:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.843099:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1788536081.843100:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1788536081.843102:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1788536081.843104:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788536081.861848:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.861863:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536081.861867:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.861870:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.861878:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.861896:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536081.861904:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.861909:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.861914:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.861919:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 4 MB=0x6a9abbebac200 00000400:00000200:3.0:1788536081.861924:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.861930:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508dfd [1] + 224 00000800:00000001:3.0:1788536081.861937:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.861941:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.861944:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.861950:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536081.861957:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.861965:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536081.861986:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862004:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536081.862019:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.862023:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536081.862025:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.862027:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.862038:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.862041:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536081.862045:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.862048:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.862051:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.862054:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac180 00000400:00000200:3.0:1788536081.862057:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536081.862061:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508ded [1] + 224 00000800:00000001:3.0:1788536081.862065:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.862071:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536081.862073:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536081.862076:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536081.862079:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.862083:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536081.862093:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862100:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536081.862103:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536081.862106:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862120:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.862124:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788536081.862130:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536081.862132:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536081.862134:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b688. 00000400:00000200:3.0:1788536081.862139:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1dd0 00000400:00000010:3.0:1788536081.862141:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1dd0. 00000100:00000001:3.0:1788536081.862143:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.862147:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536081.862179:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862186:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536081.862188:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862190:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536081.862193:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536081.862195:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536081.862197:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.862200:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536081.862208:0:2405:0:(import.c:2149:obd_at_measure()) add 36 to ffff9b247254ec60 time=108 v=36 (36 5 5 5) 00000100:00001000:3.0:1788536081.862212:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254ec18 time=54 v=5 (1 1 0 0) 00000100:00000001:3.0:1788536081.862215:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536081.862217:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536081.862220:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788536081.862223:0:2405:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.862227:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788536081.862229:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9b244134e800: skip recheck: last_committed 21474859064 00000100:00000001:3.0:1788536081.862231:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788536081.862233:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536081.862239:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862256:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.862259:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536081.862261:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536081.862266:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536081.862274:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2448434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676096:192.168.206.123@tcp:400:kworker.0 00000400:00000001:3.0:1788536081.862281:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536081.862282:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536081.862284:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15bdc0. 00000400:00000200:3.0:1788536081.862291:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1770 00000400:00000010:3.0:1788536081.862293:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1770. 00000100:00000001:3.0:1788536081.862296:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536081.862300:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536081.862309:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862315:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536081.862317:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862319:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536081.862320:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536081.862322:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536081.862335:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536081.862337:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536081.862342:0:2405:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b247254c460 time=108 v=60 (60 61 61 62) 00000100:00001000:3.0:1788536081.862346:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254c418 time=71 v=5 (1 2 0 0) 00000100:00000001:3.0:1788536081.862348:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536081.862350:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536081.862352:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788536081.862356:0:2405:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788536081.862363:0:2405:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.862366:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788536081.862367:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9b244134e800: skip recheck: last_committed 21474836545 00000100:00000001:3.0:1788536081.862369:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788536081.862370:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536081.862374:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862381:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536081.862383:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536081.862384:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536081.862388:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536081.862407:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2448434380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964675968:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536081.862412:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862415:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.862418:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2448434e00 x1875404964676096/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536136 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862426:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536081.862428:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536081.862432:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b24521da800. 02000000:00000001:3.0:1788536081.862435:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536081.862438:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536081.862439:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b247254e800 refcount=3 obd=lustre-OST0000-osc-ffff9b244134e800 00000020:00000001:3.0:1788536081.862442:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536081.862444:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b24722e5b00. 02000000:00000001:3.0:1788536081.862448:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536081.862450:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536081.862452:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2448434e00. 00000100:00000001:3.0:1788536081.862455:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536081.862457:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862458:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536081.862462:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2448434380 x1875404964675968/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536166 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536081.862468:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536081.862469:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536081.862471:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2450f5f600. 02000000:00000001:3.0:1788536081.862473:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536081.862474:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536081.862475:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b247254c000 refcount=3 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000020:00000001:3.0:1788536081.862477:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536081.862479:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b24722e5200. 02000000:00000001:3.0:1788536081.862481:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536081.862482:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536081.862483:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2448434380. 00000100:00000001:3.0:1788536081.862488:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536081.862490:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862492:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536081.862499:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536081.862501:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536081.862503:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536081.862505:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536082.790009:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536082.790030:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536082.790036:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536082.790037:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536082.790107:0:242350:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536082.790110:0:242350:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536082.790113:0:242350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536083.817243:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536083.817250:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536083.817256:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536083.817258:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536083.818493:0:236366:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536083.818498:0:236366:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536083.818502:0:236366:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536084.837843:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536084.837850:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536084.837855:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536084.837856:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536084.837918:0:248051:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536084.837921:0:248051:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536084.837925:0:248051:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788536085.733706:0:260093:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5b30 (cpu 3) has reached its deadline 4305381651, at 4305381696 00000100:00000010:3.0:1788536085.733721:0:260093:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b24480ed5a0. 00000100:00000040:2.0:1788536085.734946:0:265135:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5a20 (cpu 2) has reached its deadline 4305381651, at 4305381697 00000100:00000010:2.0:1788536085.734956:0:265135:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b24480ed9c0. 00010000:00000001:2.0:1788536085.861764:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536085.861770:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536085.861776:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536085.861777:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788536085.862036:0:252172:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536085.862040:0:252172:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536085.862042:0:252172:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536086.887286:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536086.887293:0:265135:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788536086.887295:0:265135:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788536086.887299:0:265135:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536086.887303:0:265135:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788536086.887305:0:265135:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536086.887307:0:265135:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536086.887310:0:265135:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536086.887313:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536086.887315:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536086.887747:0:236365:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536086.887750:0:236365:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536086.887754:0:236365:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788536086.950752:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) f1e73104-a124-445d-9e99-edaf7c361d90->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788536086.950762:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788536086.950770:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2472a44380. 00000020:00000040:3.0:1788536086.950776:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b244a7c6000 refcount=4 obd=MGC192.168.206.123@tcp 00000100:00000001:3.0:1788536086.950780:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788536086.950783:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788536086.950788:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788536086.950795:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b235f34d500. 00000100:00000001:3.0:1788536086.950802:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950813:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging f1e73104-a124-445d-9e99-edaf7c361d90->MGS req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788536086.950825:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788536086.950828:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950834:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d68bb55] to pc [ptlrpcd_01_00+0] req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.950855:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950858:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788536086.950861:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788536086.950863:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2472a45880. 00000020:00000040:3.0:1788536086.950865:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b2472549000 refcount=6 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000100:00000001:3.0:1788536086.950867:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788536086.950868:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788536086.950870:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788536086.950882:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b235f34d700. 00000100:00000001:3.0:1788536086.950886:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950890:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788536086.950897:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788536086.950899:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950903:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a7b2e174] to pc [ptlrpcd_01_01+1] req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.950911:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950913:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788536086.950916:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788536086.950918:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2472a45500. 00000020:00000040:3.0:1788536086.950920:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b247254c000 refcount=4 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000100:00000001:3.0:1788536086.950922:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788536086.950924:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788536086.950926:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788536086.950929:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b235f34dc00. 00000100:00000001:3.0:1788536086.950931:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950935:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788536086.950942:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788536086.950944:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950948:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000031cfad8b] to pc [ptlrpcd_01_00+0] req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.950955:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950957:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1788536086.950960:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1788536086.950963:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2472a47b80. 00000020:00000040:3.0:1788536086.950965:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b247254e800 refcount=4 obd=lustre-OST0000-osc-ffff9b244134e800 00000100:00000001:3.0:1788536086.950967:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1788536086.950973:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1788536086.950975:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1788536086.950979:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b235f34d900. 00000100:00000001:3.0:1788536086.950982:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950986:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1788536086.950993:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1788536086.950995:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.950998:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005063597c] to pc [ptlrpcd_01_01+1] req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951005:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.951007:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1788536086.951010:0:278585:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1788536086.951015:0:278585:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (10720) 00000100:00000001:3.0:1788536086.951029:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.951033:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536086.951035:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536086.951039:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951047:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536086.951049:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951053:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536086.951055:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536086.951058:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536086.951062:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2472a44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f1e73104-a124-445d-9e99-edaf7c361d90:2405:1875404964676224:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536086.951066:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536086.951070:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536086.951072:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951075:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536086.951079:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2447f49800. 02000000:00000001:3.0:1788536086.951081:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536086.951092:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536086.951095:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15bdc0. 00000400:00000010:3.0:1788536086.951103:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1770. 00000100:00000200:3.0:1788536086.951109:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676224, portal 25 00000100:00000001:3.0:1788536086.951112:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536086.951114:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179577336704 : -110894132214912 : ffff9b2472a44380) 00000100:00000040:3.0:1788536086.951120:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951128:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536086.951130:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536086.951133:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1dd0. 00000100:00000200:3.0:1788536086.951135:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875404964676224, offset 0 00000400:00000200:3.0:1788536086.951141:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951149:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951156:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536086.951161:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536086.951165:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536086.951169:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536086.951171:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951175:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539859:539859:256:4294967295] 192.168.206.123@tcp LPNI seq info [539810:539810:8:4294967295] 00000400:00000200:3.0:1788536086.951179:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951188:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536086.951194:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536086.951199:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34d200. 00000800:00000200:3.0:1788536086.951205:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536086.951212:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536086.951230:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34d200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536086.951240:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536086.951244:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536086.951246:0:2405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 2 is 4305386768 00000100:00000010:3.0:1788536086.951251:0:2405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b24480ed5a0. 00000100:00000001:3.0:1788536086.951262:0:2405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1788536086.951267:0:2405:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 3 is 4305386768 00000100:00000010:3.0:1788536086.951270:0:2405:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b24480ed300. 00000100:00000001:3.0:1788536086.951272:0:2405:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1788536086.951275:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.951278:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951280:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536086.951284:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951292:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536086.951293:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951295:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536086.951297:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536086.951299:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536086.951302:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2472a45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676480:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536086.951305:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536086.951308:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536086.951309:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951311:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536086.951314:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2447f48400. 02000000:00000001:3.0:1788536086.951316:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536086.951319:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536086.951322:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b688. 00000400:00000010:3.0:1788536086.951325:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd13b8. 00000100:00000200:3.0:1788536086.951328:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676480, portal 10 00000100:00000001:3.0:1788536086.951330:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536086.951331:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179577341184 : -110894132210432 : ffff9b2472a45500) 00000100:00000040:3.0:1788536086.951338:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951344:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536086.951346:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536086.951349:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd12a8. 00000100:00000200:3.0:1788536086.951356:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964676480, offset 0 00000400:00000200:3.0:1788536086.951361:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951364:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951368:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536086.951370:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536086.951373:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536086.951375:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536086.951377:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951380:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539860:539860:255:4294967295] 192.168.206.123@tcp LPNI seq info [539811:539811:7:4294967295] 00000400:00000200:3.0:1788536086.951384:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951391:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536086.951395:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536086.951398:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34de00. 00000800:00000200:3.0:1788536086.951402:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536086.951406:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536086.951408:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34de00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536086.951411:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.951414:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: using a timeout of 20280 usecs (21 jiffies) 00000400:00000200:3.0:1788536086.951417:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536086.951419:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305382934 00000100:00000001:3.0:1788536086.951421:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951424:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305382934 00000100:00000001:3.0:1788536086.951426:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951428:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.951430:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951432:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951434:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.951440:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536086.951442:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1788536086.951445:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.951450:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536086.951452:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536086.951455:0:2405:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:3.0:1788536086.951456:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.951459:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536086.951460:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.951462:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.951463:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1788536086.951464:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536086.951469:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951475:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536086.951477:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951479:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536086.951481:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536086.951483:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536086.951485:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2472a45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676352:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536086.951488:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536086.951490:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536086.951492:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951493:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536086.951496:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2447f48600. 02000000:00000001:3.0:1788536086.951498:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536086.951501:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536086.951503:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b630. 00000400:00000010:3.0:1788536086.951507:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1198. 00000100:00000200:3.0:1788536086.951509:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676352, portal 10 00000100:00000001:3.0:1788536086.951511:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536086.951512:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179577342080 : -110894132209536 : ffff9b2472a45880) 00000100:00000040:3.0:1788536086.951518:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951524:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536086.951526:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536086.951529:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1660. 00000100:00000200:3.0:1788536086.951535:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964676352, offset 0 00000400:00000200:3.0:1788536086.951539:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951543:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951546:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536086.951549:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536086.951551:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536086.951554:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536086.951556:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951559:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539861:539861:254:4294967295] 192.168.206.123@tcp LPNI seq info [539812:539812:6:4294967295] 00000400:00000200:3.0:1788536086.951562:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951568:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536086.951572:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536086.951576:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34db00. 00000800:00000200:3.0:1788536086.951579:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536086.951583:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536086.951586:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34db00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536086.951589:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.951592:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: using a timeout of 19380 usecs (20 jiffies) 00000400:00000200:3.0:1788536086.951595:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536086.951596:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305382933 00000100:00000001:3.0:1788536086.951599:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951601:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305382933 00000100:00000001:3.0:1788536086.951632:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951634:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.951636:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951638:0:2405:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1788536086.951641:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951719:0:2405:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1788536086.951726:0:2405:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951728:0:2405:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1788536086.951730:0:2405:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1788536086.951732:0:2405:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1788536086.951735:0:2405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2472a47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676608:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536086.951739:0:2405:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1788536086.951741:0:2405:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1788536086.951742:0:2405:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.951744:0:2405:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1788536086.951747:0:2405:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b2447f48800. 02000000:00000001:3.0:1788536086.951749:0:2405:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536086.951752:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:3.0:1788536086.951755:0:2405:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b244b15b6e0. 00000400:00000010:3.0:1788536086.951758:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd14c8. 00000100:00000200:3.0:1788536086.951761:0:2405:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676608, portal 4 00000100:00000001:3.0:1788536086.951763:0:2405:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1788536086.951764:0:2405:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179577351040 : -110894132200576 : ffff9b2472a47b80) 00000100:00000040:3.0:1788536086.951770:0:2405:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951776:0:2405:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536086.951778:0:2405:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:3.0:1788536086.951781:0:2405:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b2466dd1ee0. 00000100:00000200:3.0:1788536086.951783:0:2405:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875404964676608, offset 0 00000400:00000200:3.0:1788536086.951787:0:2405:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951791:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951794:0:2405:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:3.0:1788536086.951797:0:2405:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1788536086.951799:0:2405:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:3.0:1788536086.951802:0:2405:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1788536086.951804:0:2405:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:3.0:1788536086.951807:0:2405:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539862:539862:253:4294967295] 192.168.206.123@tcp LPNI seq info [539813:539813:5:4294967295] 00000400:00000200:3.0:1788536086.951811:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.951817:0:2405:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:3.0:1788536086.951827:0:2405:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:3.0:1788536086.951830:0:2405:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b235f34df00. 00000800:00000200:3.0:1788536086.951834:0:2405:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:3.0:1788536086.951838:0:2405:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:3.0:1788536086.951840:0:2405:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b235f34df00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1788536086.951843:0:2405:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.951846:0:2405:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: using a timeout of 116107 usecs (117 jiffies) 00000400:00000200:3.0:1788536086.951849:0:2405:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:3.0:1788536086.951851:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305383030 00000100:00000001:3.0:1788536086.951853:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951855:0:2405:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305383030 00000100:00000001:3.0:1788536086.951857:0:2405:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305386768 00000100:00000001:3.0:1788536086.951859:0:2405:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.951861:0:2405:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951864:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951866:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951877:0:2406:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.951879:0:2406:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788536086.951914:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536086.951917:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34d200. 00000400:00000200:3.0:1788536086.951922:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.951926:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536086.951930:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1dd0 00000400:00000010:3.0:1788536086.951932:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1dd0. 00000100:00000001:3.0:1788536086.951936:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536086.951941:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951950:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.951953:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.951959:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.951967:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536086.951979:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536086.951982:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34de00. 00000400:00000200:3.0:1788536086.951984:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.951987:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536086.951990:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd12a8 00000400:00000010:3.0:1788536086.951992:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd12a8. 00000100:00000001:3.0:1788536086.951994:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536086.951998:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952004:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.952007:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952012:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.952013:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536086.952019:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536086.952021:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34db00. 00000400:00000200:3.0:1788536086.952023:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.952026:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536086.952028:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1660 00000400:00000010:3.0:1788536086.952030:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1660. 00000100:00000001:3.0:1788536086.952031:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536086.952035:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952042:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.952044:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952050:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.952051:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536086.952103:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536086.952106:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b235f34df00. 00000400:00000200:3.0:1788536086.952108:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.952112:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536086.952120:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1ee0 00000400:00000010:3.0:1788536086.952122:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1ee0. 00000100:00000001:3.0:1788536086.952124:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536086.952128:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952134:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.952137:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.952142:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.952144:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:3.0:1788536086.968793:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968808:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536086.968812:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968815:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.968823:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968831:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536086.968839:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968863:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968869:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968873:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 25 MB=0x6a9abbebac280 00000400:00000200:3.0:1788536086.968877:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968883:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e0d [1] + 224 00000800:00000001:3.0:1788536086.968890:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968894:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.968898:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.968902:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536086.968909:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.968917:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536086.968926:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.968941:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536086.968960:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968964:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536086.968966:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968968:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.968971:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968974:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536086.968977:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968980:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968982:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968985:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac380 00000400:00000200:3.0:1788536086.968988:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.968992:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e1d [1] + 224 00000800:00000001:3.0:1788536086.968995:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.968998:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.969011:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.969015:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536086.969018:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969023:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536086.969030:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969036:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536086.969040:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969043:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536086.969044:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969046:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.969049:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969052:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536086.969056:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969057:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969060:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969063:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac300 00000400:00000200:3.0:1788536086.969065:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969076:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e2d [1] + 224 00000800:00000001:3.0:1788536086.969080:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969084:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.969086:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.969089:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536086.969092:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969096:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536086.969104:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969111:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536086.969114:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969117:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536086.969118:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969120:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.969123:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969126:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536086.969130:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969132:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969135:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969137:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 4 MB=0x6a9abbebac400 00000400:00000200:3.0:1788536086.969141:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536086.969144:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e3d [1] + 224 00000800:00000001:3.0:1788536086.969148:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969152:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536086.969154:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536086.969157:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536086.969160:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969163:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536086.969170:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969182:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536086.969186:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536086.969188:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.969201:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.969204:0:2405:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1788536086.969210:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536086.969212:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536086.969227:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b630. 00000400:00000200:3.0:1788536086.969231:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1198 00000400:00000010:3.0:1788536086.969233:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1198. 00000100:00000001:3.0:1788536086.969237:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969241:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536086.969249:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969255:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536086.969257:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.969259:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536086.969262:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536086.969264:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536086.969267:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.969269:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536086.969277:0:2405:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b2472549460 time=113 v=60 (60 61 61 62) 00000100:00001000:3.0:1788536086.969281:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b2472549418 time=94 v=5 (1 2 0 0) 00000100:00000001:3.0:1788536086.969284:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536086.969286:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536086.969288:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788536086.969291:0:2405:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.969294:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788536086.969296:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9b244134e800: skip recheck: last_committed 30064771462 00000100:00000001:3.0:1788536086.969298:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788536086.969300:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.969304:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969323:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.969327:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536086.969329:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536086.969333:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536086.969340:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2472a45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676352:192.168.206.123@tcp:400:kworker.0 00000400:00000001:3.0:1788536086.969346:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536086.969347:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536086.969349:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b6e0. 00000400:00000200:3.0:1788536086.969351:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd14c8 00000400:00000010:3.0:1788536086.969352:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd14c8. 00000100:00000001:3.0:1788536086.969355:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969357:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536086.969366:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969372:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536086.969373:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.969375:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536086.969377:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536086.969378:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536086.969380:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.969382:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536086.969386:0:2405:0:(import.c:2149:obd_at_measure()) add 36 to ffff9b247254ec60 time=113 v=36 (36 5 5 5) 00000100:00001000:3.0:1788536086.969389:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254ec18 time=59 v=5 (1 1 0 0) 00000100:00000001:3.0:1788536086.969391:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536086.969392:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536086.969394:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788536086.969396:0:2405:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.969398:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788536086.969399:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9b244134e800: skip recheck: last_committed 21474859064 00000100:00000001:3.0:1788536086.969407:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788536086.969409:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.969413:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969420:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.969422:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536086.969423:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536086.969426:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536086.969433:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2472a47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676608:192.168.206.123@tcp:400:kworker.0 00000400:00000001:3.0:1788536086.969437:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536086.969439:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536086.969440:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15bdc0. 00000400:00000200:3.0:1788536086.969442:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd1770 00000400:00000010:3.0:1788536086.969444:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd1770. 00000100:00000001:3.0:1788536086.969446:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969450:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536086.969457:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969463:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536086.969464:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.969466:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536086.969467:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536086.969469:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536086.969470:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.969472:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536086.969475:0:2405:0:(import.c:2149:obd_at_measure()) add 5 to ffff9b244a7c6460 time=113 v=5 (5 5 5 5) 00000100:00001000:3.0:1788536086.969478:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b244a7c6418 time=94 v=5 (1 1 0 0) 00000100:00000001:3.0:1788536086.969480:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536086.969481:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536086.969483:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1788536086.969490:0:2405:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.969492:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.969496:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969503:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.969505:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536086.969506:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536086.969510:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536086.969517:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2472a44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:f1e73104-a124-445d-9e99-edaf7c361d90:2405:1875404964676224:192.168.206.123@tcp:400:kworker.0 00000400:00000001:3.0:1788536086.969523:0:2405:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1788536086.969524:0:2405:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1788536086.969526:0:2405:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b244b15b688. 00000400:00000200:3.0:1788536086.969528:0:2405:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b2466dd13b8 00000400:00000010:3.0:1788536086.969529:0:2405:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b2466dd13b8. 00000100:00000001:3.0:1788536086.969532:0:2405:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536086.969535:0:2405:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1788536086.969542:0:2405:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969549:0:2405:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1788536086.969550:0:2405:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.969552:0:2405:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:3.0:1788536086.969553:0:2405:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1788536086.969554:0:2405:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1788536086.969556:0:2405:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1788536086.969558:0:2405:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1788536086.969562:0:2405:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b247254c460 time=113 v=60 (60 61 61 62) 00000100:00001000:3.0:1788536086.969564:0:2405:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254c418 time=76 v=5 (1 2 0 0) 00000100:00000001:3.0:1788536086.969566:0:2405:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1788536086.969567:0:2405:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1788536086.969569:0:2405:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1788536086.969577:0:2405:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1788536086.969583:0:2405:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.969585:0:2405:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1788536086.969587:0:2405:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9b244134e800: skip recheck: last_committed 21474836545 00000100:00000001:3.0:1788536086.969588:0:2405:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1788536086.969590:0:2405:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1788536086.969593:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.969599:0:2405:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1788536086.969637:0:2405:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1788536086.969639:0:2405:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1788536086.971153:0:2405:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1788536086.971175:0:2405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2472a45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7c950417-9366-4528-8162-332db0623189:2405:1875404964676480:192.168.206.123@tcp:400:kworker.0 00000100:00000001:3.0:1788536086.971181:0:2405:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.971185:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.971190:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2472a45880 x1875404964676352/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.971197:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536086.971200:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536086.971204:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2447f48600. 02000000:00000001:3.0:1788536086.971207:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536086.971210:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536086.971211:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b2472549000 refcount=5 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000020:00000001:3.0:1788536086.972090:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536086.972096:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b235f34d700. 02000000:00000001:3.0:1788536086.972106:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536086.972109:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536086.972111:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2472a45880. 00000100:00000001:3.0:1788536086.972114:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536086.972122:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.972125:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.972132:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2472a47b80 x1875404964676608/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536141 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.972139:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536086.972141:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536086.972143:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2447f48800. 02000000:00000001:3.0:1788536086.972145:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536086.972146:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536086.972148:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b247254e800 refcount=3 obd=lustre-OST0000-osc-ffff9b244134e800 00000020:00000001:3.0:1788536086.972150:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536086.972151:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b235f34d900. 02000000:00000001:3.0:1788536086.972153:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536086.972154:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536086.972156:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2472a47b80. 00000100:00000001:3.0:1788536086.972159:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536086.972160:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.972162:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.972166:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2472a44380 x1875404964676224/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536102 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.972173:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536086.972174:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536086.972175:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2447f49800. 02000000:00000001:3.0:1788536086.972178:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536086.972179:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536086.972180:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b244a7c6000 refcount=3 obd=MGC192.168.206.123@tcp 00000020:00000001:3.0:1788536086.972182:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536086.972184:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b235f34d500. 02000000:00000001:3.0:1788536086.972185:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536086.972186:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536086.972188:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2472a44380. 00000100:00000001:3.0:1788536086.972190:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536086.972191:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.972194:0:2405:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536086.972197:0:2405:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2472a45500 x1875404964676480/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536171 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536086.972330:0:2405:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1788536086.972332:0:2405:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1788536086.972334:0:2405:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b2447f48400. 02000000:00000001:3.0:1788536086.972337:0:2405:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1788536086.972338:0:2405:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:3.0:1788536086.972339:0:2405:0:(genops.c:1223:class_import_put()) import ffff9b247254c000 refcount=3 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000020:00000001:3.0:1788536086.972341:0:2405:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:3.0:1788536086.972343:0:2405:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b235f34dc00. 02000000:00000001:3.0:1788536086.972345:0:2405:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1788536086.972346:0:2405:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1788536086.972347:0:2405:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2472a45500. 00000100:00000001:3.0:1788536086.972349:0:2405:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1788536086.972350:0:2405:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.972353:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1788536086.972359:0:2405:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1788536086.972361:0:2405:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536086.972363:0:2405:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536086.972366:0:2405:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1788536087.249818:0:278702:0:(llite_lib.c:4248:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536087.914238:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536087.914244:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536087.914248:0:265135:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536087.914250:0:265135:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536087.917080:0:242350:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536087.917086:0:242350:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536087.917090:0:242350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536088.933738:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536088.933745:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536088.933750:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536088.933754:0:275334:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788536088.933756:0:275334:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788536088.933763:0:275334:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788536088.933764:0:275334:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788536088.933780:0:275334:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536088.933783:0:275334:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788536088.933785:0:275334:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536088.933786:0:275334:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536088.933789:0:275334:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536088.935803:0:236366:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536088.935809:0:236366:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536088.935813:0:236366:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536089.958419:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536089.958425:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536089.958431:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536089.958432:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536089.958580:0:248051:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536089.958584:0:248051:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536089.958587:0:248051:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0:1788536090.853755:0:260093:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5b30 (cpu 3) has reached its deadline 4305386768, at 4305386816 00000100:00000010:3.0:1788536090.853770:0:260093:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b24480ed300. 00000100:00000040:2.0:1788536090.854768:0:275334:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5a20 (cpu 2) has reached its deadline 4305386768, at 4305386817 00000100:00000010:2.0:1788536090.854777:0:275334:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b24480ed5a0. 00010000:00000001:2.0:1788536090.981971:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536090.981977:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536090.981982:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536090.981984:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788536090.983743:0:252172:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536090.983749:0:252172:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536090.983753:0:252172:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536092.006834:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536092.006841:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536092.006848:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536092.006850:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788536092.007328:0:236365:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788536092.007332:0:236365:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788536092.007335:0:236365:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788536092.070156:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) f1e73104-a124-445d-9e99-edaf7c361d90->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788536092.070166:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788536092.070176:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2467ac5500. 00000020:00000040:1.0:1788536092.070181:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b244a7c6000 refcount=4 obd=MGC192.168.206.123@tcp 00000100:00000001:1.0:1788536092.070185:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788536092.070201:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788536092.070206:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788536092.070216:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b2446a1e600. 00000100:00000001:1.0:1788536092.070221:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.070232:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging f1e73104-a124-445d-9e99-edaf7c361d90->MGS req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788536092.070243:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788536092.070247:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.070253:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1caafff] to pc [ptlrpcd_00_01+1] req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.070283:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.070285:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788536092.070289:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788536092.070292:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2467ac4e00. 00000020:00000040:1.0:1788536092.070294:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b2472549000 refcount=6 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000100:00000001:1.0:1788536092.070296:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788536092.070297:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788536092.070299:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788536092.070302:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b2446a1e500. 00000100:00000001:1.0:1788536092.070318:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.070323:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0000_UUID req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788536092.070325:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.070329:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788536092.070330:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788536092.070332:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.070332:0:2404:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788536092.070338:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.070345:0:2404:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788536092.070347:0:2404:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.070351:0:2404:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788536092.070353:0:2404:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788536092.070356:0:2404:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788536092.070359:0:2404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2467ac5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f1e73104-a124-445d-9e99-edaf7c361d90:2404:1875404964676736:192.168.206.123@tcp:400:kworker.0 00000100:00000001:0.0:1788536092.070363:0:2404:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788536092.070374:0:2404:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788536092.070376:0:2404:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.070379:0:2404:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788536092.070383:0:2404:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b244a617e00. 02000000:00000001:0.0:1788536092.070385:0:2404:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788536092.070391:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:0.0:1788536092.070430:0:2404:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b245ff38058. 00000400:00000010:0.0:1788536092.070438:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b9880. 00000100:00000200:0.0:1788536092.070444:0:2404:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676736, portal 25 00000100:00000001:0.0:1788536092.070447:0:2404:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788536092.070448:0:2404:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179393316096 : -110894316235520 : ffff9b2467ac5500) 00000100:00000040:0.0:1788536092.070454:0:2404:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.070461:0:2404:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788536092.070463:0:2404:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:0.0:1788536092.070466:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b95d8. 00000100:00000200:0.0:1788536092.070469:0:2404:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875404964676736, offset 0 00000400:00000200:0.0:1788536092.070475:0:2404:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:0.0:1788536092.070482:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:0.0:1788536092.070489:0:2404:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:0.0:1788536092.070493:0:2404:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788536092.070497:0:2404:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:0.0:1788536092.070501:0:2404:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788536092.070503:0:2404:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:0.0:1788536092.070506:0:2404:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539863:539863:256:4294967295] 192.168.206.123@tcp LPNI seq info [539814:539814:8:4294967295] 00000400:00000200:0.0:1788536092.070509:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:0.0:1788536092.070519:0:2404:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:0.0:1788536092.070524:0:2404:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:0.0:1788536092.070529:0:2404:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b245ee61800. 00000800:00000200:0.0:1788536092.070535:0:2404:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:0.0:1788536092.070543:0:2404:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:0.0:1788536092.070547:0:2404:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b245ee61800 type 1, nob 320 niov 1 nkiov 1 00000100:00000040:1.0:1788536092.070554:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d3bba73] to pc [ptlrpcd_00_00+0] req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.070563:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.070565:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788536092.070568:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788536092.070571:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2467ac6300. 00000100:00000001:0.0:1788536092.070571:0:2404:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788536092.070574:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b247254c000 refcount=4 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000400:00000200:0.0:1788536092.070575:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:1.0:1788536092.070576:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788536092.070577:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 00000100:00000001:0.0:1788536092.070577:0:2404:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 2 is 4305391887 02000000:00000001:1.0:1788536092.070579:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1788536092.070580:0:2404:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b245ed9f000. 02000000:00000010:1.0:1788536092.070582:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b2446a1ef00. 00000100:00000001:1.0:1788536092.070585:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.070586:0:2404:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 2 00000100:00000040:1.0:1788536092.070589:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-MDT0001_UUID req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788536092.070591:0:2404:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS new deadline estimation for cpu 3 is 4305391887 00000100:00000010:0.0:1788536092.070593:0:2404:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff9b245ed9f360. 00000100:00000001:1.0:1788536092.070596:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1788536092.070596:0:2404:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.123@tcp: PM QoS request now active for cpu 3 00000100:00000001:1.0:1788536092.070598:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.070599:0:2404:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788536092.070640:0:2404:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.070645:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.070647:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.070653:0:2404:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788536092.070655:0:2404:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788536092.070658:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.070659:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788536092.070661:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788536092.070663:0:2404:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788536092.070664:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.070666:0:2404:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788536092.070668:0:2404:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.070669:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.070670:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788536092.070671:0:2404:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788536092.070675:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.070682:0:2404:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788536092.070683:0:2404:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.070685:0:2404:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788536092.070687:0:2404:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788536092.070689:0:2404:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788536092.070691:0:2404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2467ac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7c950417-9366-4528-8162-332db0623189:2404:1875404964676864:192.168.206.123@tcp:400:kworker.0 00000100:00000001:0.0:1788536092.070694:0:2404:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788536092.070702:0:2404:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788536092.070703:0:2404:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.070705:0:2404:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788536092.070708:0:2404:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b244a616e00. 00000800:00000200:3.0:1788536092.070709:0:2397:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 02000000:00000001:0.0:1788536092.070710:0:2404:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788536092.070713:0:2397:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b245ee61800. 00000400:00000200:0.0:1788536092.070713:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:0.0:1788536092.070716:0:2404:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b245ff38370. 00000400:00000200:3.0:1788536092.070718:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000010:0.0:1788536092.070719:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b9220. 00000400:00000200:3.0:1788536092.070722:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536092.070727:0:2397:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b95d8 00000400:00000010:3.0:1788536092.070730:0:2397:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b95d8. 00000100:00000200:0.0:1788536092.070732:0:2404:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676864, portal 10 00000100:00000001:3.0:1788536092.070733:0:2397:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788536092.070734:0:2404:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788536092.070735:0:2404:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179393314304 : -110894316237312 : ffff9b2467ac4e00) 00000100:00000200:3.0:1788536092.070737:0:2397:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788536092.070741:0:2404:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.070745:0:2397:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1788536092.070747:0:2404:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1788536092.070748:0:2397:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788536092.070749:0:2404:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:0.0:1788536092.070751:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b9e58. 00000100:00000200:0.0:1788536092.070753:0:2404:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964676864, offset 0 00000400:00000200:0.0:1788536092.070758:0:2404:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000100:00000001:3.0:1788536092.070759:0:2397:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536092.070761:0:2397:0:(events.c:68:request_out_callback()) Process leaving 00000400:00000200:0.0:1788536092.070762:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:0.0:1788536092.070765:0:2404:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000400:00000200:0.0:1788536092.070767:0:2404:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788536092.070770:0:2404:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:0.0:1788536092.070772:0:2404:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788536092.070774:0:2404:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:0.0:1788536092.070776:0:2404:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539864:539864:256:4294967295] 192.168.206.123@tcp LPNI seq info [539815:539815:8:4294967295] 00000400:00000200:0.0:1788536092.070780:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:0.0:1788536092.070786:0:2404:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:0.0:1788536092.070790:0:2404:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:0.0:1788536092.070794:0:2404:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b245ee61600. 00000800:00000200:0.0:1788536092.070797:0:2404:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000800:00000200:0.0:1788536092.070801:0:2404:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000800:00000200:0.0:1788536092.070804:0:2404:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b245ee61600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788536092.070818:0:2404:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788536092.070821:0:2404:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: using a timeout of 19381 usecs (20 jiffies) 00000400:00000200:0.0:1788536092.070825:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:0.0:1788536092.070827:0:2404:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305388053 00000100:00000001:0.0:1788536092.070829:0:2404:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305391887 00000100:00000001:0.0:1788536092.070831:0:2404:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305388053 00000100:00000001:0.0:1788536092.070833:0:2404:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305391887 00000100:00000001:0.0:1788536092.070835:0:2404:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000800:00000200:3.0:1788536092.070837:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000100:00000001:0.0:1788536092.070837:0:2404:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000010:3.0:1788536092.070839:0:2398:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b245ee61600. 00000100:00000001:0.0:1788536092.070840:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536092.070842:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:0.0:1788536092.070842:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1788536092.070845:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536092.070848:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b9e58 00000400:00000010:3.0:1788536092.070850:0:2398:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b9e58. 00000100:00000001:0.0:1788536092.070851:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1788536092.070852:0:2398:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1788536092.070853:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1788536092.070856:0:2398:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.070864:0:2398:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536092.070867:0:2398:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.070873:0:2398:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536092.070874:0:2398:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:1.0:1788536092.071755:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000022961a90] to pc [ptlrpcd_00_01+1] req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.071776:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.071779:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788536092.071782:0:278585:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788536092.071786:0:278585:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff9b2467ac4a80. 00000020:00000040:1.0:1788536092.071789:0:278585:0:(genops.c:1210:class_import_get()) import ffff9b247254e800 refcount=4 obd=lustre-OST0000-osc-ffff9b244134e800 00000100:00000001:1.0:1788536092.071792:0:278585:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788536092.071801:0:278585:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788536092.071805:0:278585:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.071808:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 02000000:00000010:1.0:1788536092.071810:0:278585:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff9b2446a1ee00. 00000100:00100000:0.0:1788536092.071811:0:2403:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:0.0:1788536092.071812:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.071813:0:278585:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.071817:0:2403:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788536092.071819:0:2403:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788536092.071820:0:278585:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7c950417-9366-4528-8162-332db0623189->lustre-OST0000_UUID req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1788536092.071821:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.071822:0:2403:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788536092.071824:0:2403:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1788536092.071829:0:278585:0:(jobid.c:959:lustre_get_jobid()) Process entered 00000100:00000040:0.0:1788536092.071829:0:2403:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.071832:0:278585:0:(jobid.c:999:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.071836:0:278585:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007205fe6f] to pc [ptlrpcd_00_00+0] req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.071836:0:2403:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788536092.071838:0:2403:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.071841:0:2403:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788536092.071843:0:2403:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788536092.071844:0:278585:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.071845:0:2403:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.071846:0:278585:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1788536092.071848:0:2403:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2467ac6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7c950417-9366-4528-8162-332db0623189:2403:1875404964676992:192.168.206.123@tcp:400:kworker.0 00000100:00000040:1.0:1788536092.071849:0:278585:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7c950417-9366-4528-8162-332db0623189->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:0.0:1788536092.071852:0:2403:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000040:1.0:1788536092.071854:0:278585:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (10725) 02000000:00000001:0.0:1788536092.071855:0:2403:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788536092.071857:0:2403:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.071859:0:2403:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788536092.071867:0:2403:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b244a616400. 02000000:00000001:0.0:1788536092.071869:0:2403:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788536092.071873:0:2403:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000010:0.0:1788536092.071876:0:2403:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b245ff389a0. 00000400:00000010:0.0:1788536092.071880:0:2403:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b92a8. 00000100:00000200:0.0:1788536092.071885:0:2403:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964676992, portal 10 00000100:00000001:1.0:1788536092.071886:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.071887:0:2403:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1788536092.071888:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788536092.071888:0:2403:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179393319680 : -110894316231936 : ffff9b2467ac6300) 00000100:00000001:1.0:1788536092.071890:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788536092.071892:0:2404:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1788536092.071893:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788536092.071895:0:2403:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.071896:0:2404:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788536092.071898:0:2404:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.071900:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.071901:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788536092.071902:0:2404:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000001:0.0:1788536092.071902:0:2403:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788536092.071904:0:2403:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000100:00000040:1.0:1788536092.071906:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000010:0.0:1788536092.071907:0:2403:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24456b9990. 00000100:00000200:0.0:1788536092.071909:0:2403:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875404964676992, offset 0 00000100:00000001:1.0:1788536092.071912:0:2404:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1788536092.071913:0:2404:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788536092.071915:0:2404:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1788536092.071917:0:2404:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1788536092.071919:0:2404:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1788536092.071921:0:2404:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff9b2467ac4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7c950417-9366-4528-8162-332db0623189:2404:1875404964677120:192.168.206.123@tcp:400:kworker.0 00000400:00000200:0.0:1788536092.071925:0:2403:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:0.0:1788536092.071931:0:2403:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:1.0:1788536092.071932:0:2404:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1788536092.071935:0:2404:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:0.0:1788536092.071936:0:2403:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 02000000:00000001:1.0:1788536092.071937:0:2404:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788536092.071938:0:2404:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000200:0.0:1788536092.071940:0:2403:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 02000000:00000010:1.0:1788536092.071941:0:2404:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff9b24521da800. 02000000:00000001:1.0:1788536092.071942:0:2404:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788536092.071943:0:2403:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:1.0:1788536092.071946:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:0.0:1788536092.071946:0:2403:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000010:1.0:1788536092.071949:0:2404:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff9b24484d17e8. 00000400:00000200:0.0:1788536092.071949:0:2403:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000010:1.0:1788536092.071953:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24460f42a8. 00000400:00000200:0.0:1788536092.071953:0:2403:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539865:539865:256:4294967295] 192.168.206.123@tcp LPNI seq info [539816:539816:8:4294967295] 00000100:00000200:1.0:1788536092.071956:0:2404:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875404964677120, portal 4 00000100:00000001:1.0:1788536092.071957:0:2404:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000400:00000200:0.0:1788536092.071957:0:2403:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:1.0:1788536092.071959:0:2404:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446633179393313408 : -110894316238208 : ffff9b2467ac4a80) 00000100:00000040:1.0:1788536092.071963:0:2404:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:0.0:1788536092.071965:0:2403:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000100:00000001:1.0:1788536092.071970:0:2404:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000800:00000200:0.0:1788536092.071970:0:2403:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000100:00000040:1.0:1788536092.071972:0:2404:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.123@tcp 00000400:00000010:1.0:1788536092.071974:0:2404:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff9b24460f4cc0. 00000800:00000010:0.0:1788536092.071974:0:2403:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b245ee61600. 00000100:00000200:1.0:1788536092.071976:0:2404:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875404964677120, offset 0 00000800:00000200:0.0:1788536092.071978:0:2403:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000400:00000200:1.0:1788536092.071980:0:2404:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.123@tcp 00000400:00000200:1.0:1788536092.071983:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000800:00000200:0.0:1788536092.071983:0:2403:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000400:00000200:1.0:1788536092.071986:0:2404:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.123@tcp local destination 00000800:00000200:0.0:1788536092.071987:0:2403:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b245ee61600 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1788536092.071989:0:2404:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1788536092.071991:0:2404:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.23@tcp 00000400:00000200:1.0:1788536092.071994:0:2404:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.23@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1788536092.071996:0:2404:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.123@tcp 00000400:00000200:1.0:1788536092.071998:0:2404:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.23@tcp NI seq info: [539866:539866:255:4294967295] 192.168.206.123@tcp LPNI seq info [539817:539817:7:4294967295] 00000400:00000200:1.0:1788536092.072002:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:0.0:1788536092.072003:0:2403:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788536092.072006:0:2403:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: using a timeout of 20280 usecs (21 jiffies) 00000400:00000200:1.0:1788536092.072008:0:2404:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp:) -> 192.168.206.123@tcp(192.168.206.123@tcp:192.168.206.123@tcp) : PUT try# 0 00000800:00000200:1.0:1788536092.072012:0:2404:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.123@tcp 00000800:00000010:1.0:1788536092.072015:0:2404:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff9b2446a1e200. 00000800:00000200:1.0:1788536092.072018:0:2404:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff9b2449f80300] -> 12345-192.168.206.123@tcp (5) 00000400:00000200:0.0:1788536092.072022:0:2403:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:0.0:1788536092.072024:0:2403:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305388055 00000800:00000200:1.0:1788536092.072027:0:2404:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.123@tcp ip 192.168.206.123:988 00000100:00000001:0.0:1788536092.072027:0:2403:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305391887 00000800:00000200:1.0:1788536092.072029:0:2404:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff9b2446a1e200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788536092.072029:0:2403:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305388055 00000100:00000001:1.0:1788536092.072032:0:2404:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.072032:0:2403:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305391887 00000100:00000001:0.0:1788536092.072034:0:2403:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000040:1.0:1788536092.072035:0:2404:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: using a timeout of 116027 usecs (117 jiffies) 00000100:00000001:0.0:1788536092.072036:0:2403:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1788536092.072038:0:2404:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000100:00000001:0.0:1788536092.072038:0:2403:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.072040:0:2404:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 2 is 4305388151 00000100:00000001:0.0:1788536092.072041:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.072043:0:2404:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 2, keep current deadline 4305391887 00000100:00000001:0.0:1788536092.072043:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.072044:0:2403:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788536092.072045:0:2404:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS new deadline estimation for cpu 3 is 4305388151 00000100:00000001:0.0:1788536092.072045:0:2403:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.072047:0:2404:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff9b244134e800: PM QoS request active for cpu 3, keep current deadline 4305391887 00000100:00000001:0.0:1788536092.072047:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.072049:0:2404:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788536092.072050:0:2404:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.072052:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1788536092.072118:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536092.072121:0:2398:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b245ee61600. 00000400:00000200:3.0:1788536092.072127:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536092.072132:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536092.072136:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b9990 00000400:00000010:3.0:1788536092.072138:0:2398:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b9990. 00000100:00000001:3.0:1788536092.072141:0:2398:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536092.072145:0:2398:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.072153:0:2398:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536092.072167:0:2398:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.072173:0:2398:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536092.072174:0:2398:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1788536092.072226:0:2398:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1788536092.072233:0:2398:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff9b2446a1e200. 00000400:00000200:3.0:1788536092.072236:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536092.072240:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000400:00000200:3.0:1788536092.072243:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24460f4cc0 00000400:00000010:3.0:1788536092.072244:0:2398:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24460f4cc0. 00000100:00000001:3.0:1788536092.072247:0:2398:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1788536092.072250:0:2398:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.072258:0:2398:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1788536092.072260:0:2398:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.072266:0:2398:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1788536092.072267:0:2398:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:1.0:1788536092.073289:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788536092.075508:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.075524:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788536092.075528:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.075530:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.075538:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.075546:0:2397:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:2.0:1788536092.075554:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.075558:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.075563:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.075567:0:2397:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 25 MB=0x6a9abbebac480 00000400:00000200:2.0:1788536092.075572:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.075583:0:2397:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e4d [1] + 224 00000800:00000001:2.0:1788536092.075589:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.075596:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.075635:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788536092.075640:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:2.0:1788536092.075658:0:2397:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788536092.075667:0:2397:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788536092.075682:0:2397:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536092.075702:0:2397:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788536092.075711:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.075714:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.076164:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.076169:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788536092.076176:0:2404:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788536092.076178:0:2404:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788536092.076182:0:2404:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b245ff38058. 00000400:00000200:1.0:1788536092.076204:0:2404:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b9880 00000400:00000010:1.0:1788536092.076207:0:2404:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b9880. 00000100:00000001:1.0:1788536092.076210:0:2404:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788536092.076216:0:2404:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788536092.076225:0:2404:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.076232:0:2404:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788536092.076233:0:2404:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.076236:0:2404:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788536092.076239:0:2404:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788536092.076241:0:2404:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788536092.076242:0:2404:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788536092.076245:0:2404:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788536092.076250:0:2404:0:(import.c:2149:obd_at_measure()) add 5 to ffff9b244a7c6460 time=119 v=5 (5 5 5 5) 00000100:00001000:1.0:1788536092.076260:0:2404:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b244a7c6418 time=100 v=5 (1 1 0 0) 00000100:00000001:1.0:1788536092.076263:0:2404:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788536092.076264:0:2404:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788536092.076267:0:2404:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788536092.076269:0:2404:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.076271:0:2404:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.076276:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.076283:0:2404:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788536092.076286:0:2404:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788536092.076287:0:2404:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788536092.076291:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788536092.076298:0:2404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2467ac5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:f1e73104-a124-445d-9e99-edaf7c361d90:2404:1875404964676736:192.168.206.123@tcp:400:kworker.0 00000100:00000001:1.0:1788536092.076302:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.076305:0:2404:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788536092.076308:0:2404:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2467ac5500 x1875404964676736/t0(0) o400->MGC192.168.206.123@tcp@192.168.206.123@tcp:26/25 lens 224/224 e 0 to 0 dl 1788536108 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.076315:0:2404:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788536092.076318:0:2404:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788536092.076321:0:2404:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b244a617e00. 02000000:00000001:1.0:1788536092.076325:0:2404:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788536092.076327:0:2404:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788536092.076329:0:2404:0:(genops.c:1223:class_import_put()) import ffff9b244a7c6000 refcount=3 obd=MGC192.168.206.123@tcp 00000020:00000001:1.0:1788536092.076331:0:2404:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788536092.076342:0:2404:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b2446a1e600. 02000000:00000001:1.0:1788536092.076346:0:2404:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788536092.076348:0:2404:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788536092.076350:0:2404:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2467ac5500. 00000100:00000001:1.0:1788536092.076353:0:2404:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788536092.076354:0:2404:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.076357:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.076358:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.076365:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788536092.076367:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.076368:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:3.0:1788536092.081649:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536092.081665:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:3.0:1788536092.081668:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536092.081671:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536092.081679:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536092.081686:0:2398:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:3.0:1788536092.081694:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536092.081698:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536092.081703:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536092.081707:0:2398:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac500 00000400:00000200:3.0:1788536092.081711:0:2398:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:3.0:1788536092.081717:0:2398:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e55 [1] + 224 00000800:00000001:3.0:1788536092.081724:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536092.081730:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:3.0:1788536092.081734:0:2398:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1788536092.081738:0:2398:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:3.0:1788536092.081745:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1788536092.081753:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:3.0:1788536092.081763:0:2398:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1788536092.081797:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:3.0:1788536092.081806:0:2398:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:3.0:1788536092.081809:0:2398:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.081924:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.081928:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788536092.081935:0:2404:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788536092.081937:0:2404:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788536092.081940:0:2404:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b245ff38370. 00000400:00000200:1.0:1788536092.081945:0:2404:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b9220 00000400:00000010:1.0:1788536092.081947:0:2404:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b9220. 00000100:00000001:1.0:1788536092.081950:0:2404:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788536092.081955:0:2404:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788536092.081964:0:2404:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.081970:0:2404:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788536092.081972:0:2404:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.081974:0:2404:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788536092.081977:0:2404:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788536092.081979:0:2404:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788536092.081981:0:2404:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788536092.081983:0:2404:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788536092.081991:0:2404:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b2472549460 time=119 v=60 (60 61 61 62) 00000100:00001000:1.0:1788536092.081996:0:2404:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b2472549418 time=100 v=5 (1 2 0 0) 00000100:00000001:1.0:1788536092.081998:0:2404:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788536092.081999:0:2404:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788536092.082002:0:2404:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788536092.082005:0:2404:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.082008:0:2404:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788536092.082010:0:2404:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff9b244134e800: skip recheck: last_committed 30064771462 00000100:00000001:1.0:1788536092.082012:0:2404:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788536092.082014:0:2404:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.082019:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.082027:0:2404:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788536092.082036:0:2404:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788536092.082038:0:2404:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788536092.082044:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788536092.082052:0:2404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2467ac4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7c950417-9366-4528-8162-332db0623189:2404:1875404964676864:192.168.206.123@tcp:400:kworker.0 00000100:00000001:1.0:1788536092.082114:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.082116:0:2404:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788536092.082120:0:2404:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2467ac4e00 x1875404964676864/t0(0) o400->lustre-MDT0000-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.082126:0:2404:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788536092.082129:0:2404:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788536092.082132:0:2404:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b244a616e00. 02000000:00000001:1.0:1788536092.082136:0:2404:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788536092.082138:0:2404:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788536092.082139:0:2404:0:(genops.c:1223:class_import_put()) import ffff9b2472549000 refcount=5 obd=lustre-MDT0000-mdc-ffff9b244134e800 00000020:00000001:1.0:1788536092.082141:0:2404:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788536092.082144:0:2404:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b2446a1e500. 02000000:00000001:1.0:1788536092.082147:0:2404:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788536092.082148:0:2404:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788536092.082150:0:2404:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2467ac4e00. 00000100:00000001:1.0:1788536092.082153:0:2404:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788536092.082154:0:2404:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.082157:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.082159:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.082160:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788536092.082162:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.082164:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1788536092.088500:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088519:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788536092.088523:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088526:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.088534:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088541:0:2397:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:2.0:1788536092.088551:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088555:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088560:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088564:0:2397:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 10 MB=0x6a9abbebac580 00000400:00000200:2.0:1788536092.088569:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088575:0:2397:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e5d [1] + 224 00000800:00000001:2.0:1788536092.088581:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088584:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.088587:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788536092.088592:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:2.0:1788536092.088598:0:2397:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788536092.088620:0:2397:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788536092.088631:0:2397:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1788536092.088654:0:2397:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1788536092.088662:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088679:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1788536092.088681:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088683:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.088686:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088690:0:2397:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.23@tcp(192.168.206.23@tcp) <- 192.168.206.123@tcp : PUT - for me 00000400:00000200:2.0:1788536092.088693:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088696:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088698:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088701:0:2397:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.123@tcp of length 224 into portal 4 MB=0x6a9abbebac600 00000400:00000200:2.0:1788536092.088704:0:2397:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.123@tcp to cpt 1 00000400:00000200:2.0:1788536092.088708:0:2397:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.123@tcp of length 224/224 into md 0x508e65 [1] + 224 00000800:00000001:2.0:1788536092.088712:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:2.0:1788536092.088719:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1788536092.088721:0:2397:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1788536092.088725:0:2397:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.23@tcp->192.168.206.123@tcp: PUT: OK 00000100:00000001:2.0:1788536092.088728:0:2397:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1788536092.088732:0:2397:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1788536092.088741:0:2397:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.088809:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.088813:0:2404:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788536092.088815:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.088820:0:2403:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788536092.088826:0:2403:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788536092.088828:0:2403:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788536092.088831:0:2403:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b245ff389a0. 00000100:00000001:2.0:1788536092.088832:0:2397:0:(events.c:163:reply_in_callback()) Process leaving 00000400:00000200:0.0:1788536092.088838:0:2403:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24456b92a8 00000400:00000010:0.0:1788536092.088842:0:2403:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24456b92a8. 00000100:00000001:0.0:1788536092.088846:0:2403:0:(events.c:78:reply_in_callback()) Process entered 00000800:00000001:2.0:1788536092.088850:0:2397:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000200:0.0:1788536092.088852:0:2403:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1788536092.088855:0:2397:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:0.0:1788536092.088867:0:2403:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.088875:0:2403:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788536092.088877:0:2403:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.088880:0:2403:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788536092.088883:0:2403:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788536092.088885:0:2403:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788536092.088888:0:2403:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788536092.088891:0:2403:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788536092.088898:0:2403:0:(import.c:2149:obd_at_measure()) add 60 to ffff9b247254c460 time=119 v=60 (60 61 61 62) 00000100:00001000:0.0:1788536092.088902:0:2403:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254c418 time=82 v=5 (1 2 0 0) 00000100:00000001:0.0:1788536092.088911:0:2403:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788536092.088913:0:2403:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788536092.088916:0:2403:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788536092.088921:0:2403:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788536092.088928:0:2403:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.088930:0:2403:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788536092.088932:0:2403:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff9b244134e800: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1788536092.088934:0:2403:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788536092.088936:0:2403:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788536092.088940:0:2403:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.088947:0:2403:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788536092.088950:0:2403:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788536092.088951:0:2403:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788536092.088955:0:2403:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788536092.088962:0:2403:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2467ac6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7c950417-9366-4528-8162-332db0623189:2403:1875404964676992:192.168.206.123@tcp:400:kworker.0 00000100:00000001:0.0:1788536092.088967:0:2403:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.088969:0:2403:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788536092.088972:0:2403:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2467ac6300 x1875404964676992/t0(0) o400->lustre-MDT0001-mdc-ffff9b244134e800@192.168.206.123@tcp:12/10 lens 224/224 e 0 to 0 dl 1788536177 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788536092.088978:0:2403:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788536092.088980:0:2403:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788536092.088984:0:2403:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b244a616400. 02000000:00000001:0.0:1788536092.088987:0:2403:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788536092.088989:0:2403:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788536092.088991:0:2403:0:(genops.c:1223:class_import_put()) import ffff9b247254c000 refcount=3 obd=lustre-MDT0001-mdc-ffff9b244134e800 00000020:00000001:0.0:1788536092.088993:0:2403:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788536092.088995:0:2403:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b2446a1ef00. 02000000:00000001:0.0:1788536092.088998:0:2403:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788536092.089006:0:2403:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788536092.089008:0:2403:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2467ac6300. 00000100:00000001:0.0:1788536092.089011:0:2403:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788536092.089013:0:2403:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.089016:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788536092.089022:0:2403:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788536092.089025:0:2403:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788536092.089026:0:2403:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788536092.089029:0:2403:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1788536092.100246:0:2404:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788536092.100253:0:2404:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788536092.100256:0:2404:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff9b24484d17e8. 00000400:00000200:1.0:1788536092.100264:0:2404:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff9b24460f42a8 00000400:00000010:1.0:1788536092.100267:0:2404:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff9b24460f42a8. 00000100:00000001:1.0:1788536092.100271:0:2404:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788536092.100457:0:2404:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788536092.100472:0:2404:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.100480:0:2404:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788536092.100483:0:2404:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.100486:0:2404:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788536092.100489:0:2404:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788536092.100491:0:2404:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788536092.100493:0:2404:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788536092.100496:0:2404:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788536092.100505:0:2404:0:(import.c:2149:obd_at_measure()) add 36 to ffff9b247254ec60 time=119 v=36 (36 5 5 5) 00000100:00001000:1.0:1788536092.100509:0:2404:0:(import.c:2149:obd_at_measure()) add 1 to ffff9b247254ec18 time=65 v=5 (1 1 0 0) 00000100:00000001:1.0:1788536092.100511:0:2404:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788536092.100513:0:2404:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788536092.100515:0:2404:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788536092.100518:0:2404:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.100521:0:2404:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788536092.100523:0:2404:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff9b244134e800: skip recheck: last_committed 21474859064 00000100:00000001:1.0:1788536092.100525:0:2404:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788536092.100527:0:2404:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788536092.100532:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.100539:0:2404:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788536092.100542:0:2404:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788536092.100544:0:2404:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788536092.100548:0:2404:0:(lustre_net.h:2417:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788536092.100556:0:2404:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff9b2467ac4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7c950417-9366-4528-8162-332db0623189:2404:1875404964677120:192.168.206.123@tcp:400:kworker.0 00000100:00000001:1.0:1788536092.100561:0:2404:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.100564:0:2404:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788536092.100574:0:2404:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff9b2467ac4a80 x1875404964677120/t0(0) o400->lustre-OST0000-osc-ffff9b244134e800@192.168.206.123@tcp:28/4 lens 224/224 e 0 to 0 dl 1788536147 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788536092.100580:0:2404:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788536092.100582:0:2404:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788536092.100585:0:2404:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff9b24521da800. 02000000:00000001:1.0:1788536092.100590:0:2404:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788536092.100592:0:2404:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788536092.100593:0:2404:0:(genops.c:1223:class_import_put()) import ffff9b247254e800 refcount=3 obd=lustre-OST0000-osc-ffff9b244134e800 00000020:00000001:1.0:1788536092.100596:0:2404:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788536092.100598:0:2404:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff9b2446a1ee00. 02000000:00000001:1.0:1788536092.100601:0:2404:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788536092.100618:0:2404:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788536092.100620:0:2404:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff9b2467ac4a80. 00000100:00000001:1.0:1788536092.100624:0:2404:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788536092.100625:0:2404:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.100628:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788536092.100634:0:2404:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788536092.100636:0:2404:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788536092.100639:0:2404:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788536092.100641:0:2404:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536093.029830:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536093.029836:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536093.029841:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536093.029843:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788536093.030154:0:242350:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536093.030158:0:242350:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536093.030161:0:242350:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788536094.054291:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536094.054296:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536094.054301:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536094.054302:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788536094.056406:0:236366:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788536094.056412:0:236366:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788536094.056417:0:236366:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1788536094.644529:0:278843:0:(llite_lib.c:4248:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000001:00000001:2.0:1788536094.970211:0:278879:0:(debug.c:652:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:2.0:1788536094.970217:0:278879:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_812a: @@@@@@ FAIL: client: import is not in CONNECTING state after 50 00000001:00000001:2.0:1788536094.982570:0:278879:0:(debug.c:655:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:2.0:1788536095.078322:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536095.078328:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536095.078335:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536095.078336:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788536095.079297:0:248051:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536095.079302:0:248051:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536095.079306:0:248051:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:2.0:1788536095.973676:0:275334:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5a20 (cpu 2) has reached its deadline 4305391887, at 4305391936 00000100:00000010:2.0:1788536095.973691:0:275334:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b245ed9f000. 00000100:00000040:3.0:1788536095.973859:0:260093:0:(connection.c:75:cpu_latency_work()) work item of ffff9b2443ea5b30 (cpu 3) has reached its deadline 4305391887, at 4305391936 00000100:00000010:3.0:1788536095.973865:0:260093:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff9b245ed9f360. 00010000:00000001:2.0:1788536096.101715:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536096.101722:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788536096.101726:0:275334:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788536096.101727:0:275334:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788536096.101802:0:252172:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788536096.101805:0:252172:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788536096.101808:0:252172:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1788536096.241003:0:278980:0:(llite_lib.c:4248:ll_show_options()) Process leaving (rc=0 : 0 : 0) Debug log: 2037 lines, 2037 kept, 0 dropped, 0 bad.