00010000:00000001:2.0F:1788941297.394944:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941297.395057:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941297.395064:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941297.395065:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941297.395112:0:236343:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941297.395115:0:236343:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941297.395117:0:236343:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0F:1788941298.165446:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941298.165464:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941298.165471:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772195500. 00000020:00000040:1.0:1788941298.165476:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b1000 refcount=4 obd=MGC192.168.206.112@tcp 00000100:00000001:1.0:1788941298.165481:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941298.165484:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941298.165489:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941298.165496:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083d100. 00000100:00000001:1.0:1788941298.165503:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.165515:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941298.165524:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941298.165528:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.165533:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f2224f53] to pc [ptlrpcd_00_01+1] req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0F:1788941298.165581:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.165591:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.165593:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941298.165599:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.165606:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941298.165608:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.165611:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941298.165613:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941298.165615:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.165618:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772195500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959590144:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.165622:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941298.165626:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941298.165628:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.165630:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941298.165635:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758915200. 02000000:00000001:0.0:1788941298.165636:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941298.165643:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941298.165647:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17688. 00000400:00000010:0.0:1788941298.165654:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87cc0. 00000100:00000200:0.0:1788941298.165660:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590144, portal 25 00000100:00000001:0.0:1788941298.165663:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941298.165665:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768542976 : -131290941008640 : ffff889772195500) 00000100:00000040:0.0:1788941298.165683:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.165690:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941298.165693:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941298.165696:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87a18. 00000100:00000200:0.0:1788941298.165699:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875827959590144, offset 0 00000400:00000200:0.0:1788941298.165704:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941298.165711:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.165719:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941298.165723:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941298.165727:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941298.165749:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941298.165752:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941298.165756:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545671:545671:256:4294967295] 192.168.206.112@tcp LPNI seq info [545656:545656:8:4294967295] 00000400:00000200:0.0:1788941298.165760:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.165778:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941298.165785:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941298.165790:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84100. 00000800:00000200:0.0:1788941298.165796:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941298.165802:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941298.165806:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941298.165819:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941298.165823:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941298.165826:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 0 is 4307175137 00000100:00000010:0.0:1788941298.165830:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3840. 00000100:00000001:0.0:1788941298.165837:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788941298.165843:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 1 is 4307175137 00000100:00000010:0.0:1788941298.165846:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3060. 00000100:00000001:0.0:1788941298.165849:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788941298.165887:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.165890:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.165893:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.165896:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941298.165942:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.165945:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941298.165949:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941298.165952:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772194380. 00000020:00000040:1.0:1788941298.165955:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc8a800 refcount=6 obd=lustre-MDT0000-mdc-ffff889767e18000 00000100:00000001:1.0:1788941298.165957:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941298.165959:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941298.165961:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941298.165964:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083db00. 00000100:00000001:1.0:1788941298.165968:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.165973:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941298.165981:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941298.165983:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.165987:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007952ee1c] to pc [ptlrpcd_00_00+0] req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.168521:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.168528:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788941298.168534:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.168536:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.168539:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.168543:0:2395:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:0.0:1788941298.168545:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.168547:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.168548:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.168550:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.168561:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.168563:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941298.168574:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.168583:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941298.168585:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.168588:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941298.168592:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941298.168594:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.168597:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772194380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590272:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.168601:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941298.168607:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941298.168609:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.168612:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941298.168616:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758915800. 02000000:00000001:0.0:1788941298.168619:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941298.168624:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941298.168629:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a172c0. 00000400:00000010:0.0:1788941298.168636:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87908. 00000100:00000200:0.0:1788941298.168643:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590272, portal 10 00000100:00000001:0.0:1788941298.168645:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941298.168647:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768538496 : -131290941013120 : ffff889772194380) 00000100:00000040:0.0:1788941298.168655:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.168662:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941298.168665:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941298.168668:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87dd0. 00000100:00000200:0.0:1788941298.168671:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959590272, offset 0 00000400:00000200:0.0:1788941298.168677:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941298.168684:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.168691:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941298.168695:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941298.168699:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941298.168708:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941298.168710:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941298.168714:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545672:545672:255:4294967295] 192.168.206.112@tcp LPNI seq info [545657:545657:7:4294967295] 00000400:00000200:0.0:1788941298.168719:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.168727:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941298.168733:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941298.168738:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84c00. 00000800:00000200:0.0:1788941298.168744:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941298.168752:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941298.168755:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84c00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941298.168760:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.168775:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: using a timeout of 34620 usecs (35 jiffies) 00000400:00000200:0.0:1788941298.168779:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941298.168782:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307171368 00000100:00000001:0.0:1788941298.168785:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.168787:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307171368 00000100:00000001:0.0:1788941298.168789:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.168791:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.168793:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.168796:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.168798:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788941298.168869:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941298.168872:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84100. 00000400:00000200:0.0:1788941298.168878:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.168883:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941298.168888:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87a18 00000400:00000010:0.0:1788941298.168891:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87a18. 00000100:00000001:0.0:1788941298.168895:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941298.168900:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.169650:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.169655:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.169663:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.169664:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941298.169732:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941298.169736:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84c00. 00000400:00000200:0.0:1788941298.169739:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.169743:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941298.169746:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87dd0 00000400:00000010:0.0:1788941298.169748:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87dd0. 00000100:00000001:0.0:1788941298.169750:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941298.169754:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.169761:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.169774:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.169780:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.169781:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788941298.169813:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.169816:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941298.171049:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171063:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941298.171068:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941298.171073:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772195880. 00000100:00000001:0.0:1788941298.171223:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.171227:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.171230:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.171233:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:1.0:1788941298.171270:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc88000 refcount=4 obd=lustre-MDT0001-mdc-ffff889767e18000 00000100:00000001:1.0:1788941298.171274:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941298.171276:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941298.171280:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941298.171287:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083d900. 00000100:00000001:1.0:1788941298.171293:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171301:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941298.171311:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941298.171314:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171320:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000281c4716] to pc [ptlrpcd_00_01+1] req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.171344:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171347:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941298.171350:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941298.171353:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772195180. 00000020:00000040:1.0:1788941298.171355:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b5800 refcount=4 obd=lustre-OST0000-osc-ffff889767e18000 00000100:00000001:1.0:1788941298.171357:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941298.171358:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941298.171360:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941298.171363:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083dd00. 00000100:00000001:1.0:1788941298.171366:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171370:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941298.171376:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941298.171378:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171381:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b6665cfc] to pc [ptlrpcd_00_00+0] req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.171399:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941298.171401:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788941298.171404:0:233298:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788941298.171409:0:233298:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12509) 00000100:00000001:0.0:1788941298.171460:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.171463:0:2394:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.171465:0:2394:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941298.171471:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.171478:0:2394:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941298.171480:0:2394:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.171491:0:2394:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941298.171493:0:2394:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941298.171496:0:2394:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.171500:0:2394:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772195180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2394:1875827959590528:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.171504:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941298.171508:0:2394:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941298.171510:0:2394:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.171513:0:2394:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941298.171518:0:2394:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758914a00. 02000000:00000001:0.0:1788941298.171520:0:2394:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941298.171525:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941298.171529:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17738. 00000400:00000010:0.0:1788941298.171535:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87dd0. 00000100:00000200:0.0:1788941298.171540:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590528, portal 4 00000100:00000001:0.0:1788941298.171543:0:2394:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941298.171544:0:2394:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768542080 : -131290941009536 : ffff889772195180) 00000100:00000040:0.0:1788941298.171551:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.171557:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941298.171559:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941298.171562:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87a18. 00000100:00000200:0.0:1788941298.171564:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875827959590528, offset 0 00000400:00000200:0.0:1788941298.171569:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941298.171576:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.171583:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941298.171587:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941298.171591:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941298.171594:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941298.171596:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941298.171599:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545673:545673:256:4294967295] 192.168.206.112@tcp LPNI seq info [545658:545658:8:4294967295] 00000400:00000200:0.0:1788941298.171603:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.171611:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941298.171621:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941298.171626:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84c00. 00000800:00000200:0.0:1788941298.171631:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941298.171637:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941298.171641:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84c00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941298.172997:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.173006:0:2394:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: using a timeout of 139423 usecs (140 jiffies) 00000400:00000200:0.0:1788941298.173012:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941298.173015:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307171478 00000100:00000001:0.0:1788941298.173018:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.173020:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307171478 00000100:00000001:0.0:1788941298.173022:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.173024:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.173027:0:2394:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173030:0:2394:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173033:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.173040:0:2394:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.173042:0:2394:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:0.0:1788941298.173044:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.173046:0:2394:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.173047:0:2394:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.173049:0:2394:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00000001:0.0:1788941298.173051:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.173053:0:2394:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.173054:0:2394:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.173055:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.173056:0:2394:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.173057:0:2394:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941298.173065:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.173072:0:2394:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941298.173073:0:2394:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.173085:0:2394:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941298.173088:0:2394:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941298.173090:0:2394:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941298.173093:0:2394:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772195880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2394:1875827959590400:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.173096:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941298.173100:0:2394:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941298.173102:0:2394:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.173104:0:2394:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941298.173109:0:2394:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758914400. 02000000:00000001:0.0:1788941298.173110:0:2394:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941298.173114:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941298.173118:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a170b0. 00000400:00000010:0.0:1788941298.173123:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87f68. 00000100:00000200:0.0:1788941298.173127:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590400, portal 10 00000100:00000001:0.0:1788941298.173142:0:2394:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941298.173144:0:2394:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768543872 : -131290941007744 : ffff889772195880) 00000100:00000040:0.0:1788941298.173150:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.173156:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941298.173158:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941298.173161:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87770. 00000100:00000200:0.0:1788941298.173163:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959590400, offset 0 00000400:00000200:0.0:1788941298.173168:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941298.173175:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.173181:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941298.173185:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941298.173188:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941298.173191:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941298.173193:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941298.173196:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545674:545674:255:4294967295] 192.168.206.112@tcp LPNI seq info [545659:545659:7:4294967295] 00000400:00000200:0.0:1788941298.173200:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.173207:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941298.173218:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941298.173222:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84100. 00000800:00000200:0.0:1788941298.173228:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941298.173234:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941298.173238:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941298.173242:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.173245:0:2394:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: using a timeout of 20886 usecs (21 jiffies) 00000400:00000200:0.0:1788941298.173248:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941298.173250:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307171359 00000100:00000001:0.0:1788941298.173252:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.173254:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307171359 00000100:00000001:0.0:1788941298.173255:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307175137 00000100:00000001:0.0:1788941298.173257:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.173259:0:2394:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173261:0:2394:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173263:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173281:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.173283:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941298.173285:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.173287:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788941298.175811:0:2387:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941298.175819:0:2387:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84c00. 00000400:00000200:1.0:1788941298.175828:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941298.175836:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941298.175843:0:2387:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87a18 00000400:00000010:1.0:1788941298.175845:0:2387:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87a18. 00000100:00000001:1.0:1788941298.175850:0:2387:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941298.175871:0:2387:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.175882:0:2387:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941298.175885:0:2387:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.175893:0:2387:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941298.175894:0:2387:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788941298.178456:0:2387:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941298.178465:0:2387:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84100. 00000400:00000200:1.0:1788941298.178473:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941298.178481:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941298.178487:0:2387:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87770 00000400:00000010:1.0:1788941298.178489:0:2387:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87770. 00000100:00000001:1.0:1788941298.178494:0:2387:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941298.178501:0:2387:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.178512:0:2387:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941298.178515:0:2387:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941298.178542:0:2387:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941298.178544:0:2387:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788941298.183803:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.183822:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941298.183826:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.183828:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.183837:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183845:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941298.183867:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183872:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183878:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183882:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30ed80 00000400:00000200:0.0:1788941298.183886:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183900:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x537f79 [1] + 224 00000800:00000001:0.0:1788941298.183909:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.183913:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.183916:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.183921:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941298.183927:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.183936:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941298.183947:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.183966:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941298.183975:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.183979:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941298.183980:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.183982:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.183985:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183988:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941298.183991:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183994:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183996:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.183999:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 25 MB=0x6aa0e3b30ed00 00000400:00000200:0.0:1788941298.184002:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.184006:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.112@tcp of length 224/224 into md 0x537f69 [1] + 224 00000800:00000001:0.0:1788941298.184009:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.184016:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.184018:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.184021:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941298.184024:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.184027:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941298.184035:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.184047:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941298.184051:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.184054:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.184071:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.188454:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788941298.188466:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941298.188469:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941298.188473:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a172c0. 00000400:00000200:0.0:1788941298.188481:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87908 00000400:00000010:0.0:1788941298.188483:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87908. 00000100:00000001:0.0:1788941298.188487:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.188497:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941298.188515:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188523:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941298.188525:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188527:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941298.188531:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941298.188533:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941298.188535:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.188538:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941298.188546:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc8ac60 time=96 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941298.188550:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc8ac18 time=140 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941298.188552:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941298.188554:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941298.188557:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941298.188560:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.188564:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941298.188566:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff889767e18000: skip recheck: last_committed 30064771462 00000100:00000001:0.0:1788941298.188568:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941298.188570:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.188574:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188601:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.188604:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941298.188606:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941298.188610:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941298.188618:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772194380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590272:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941298.188623:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941298.188625:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941298.188627:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17688. 00000400:00000200:0.0:1788941298.188629:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87cc0 00000400:00000010:0.0:1788941298.188630:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87cc0. 00000100:00000001:0.0:1788941298.188633:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.188636:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941298.188645:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188651:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941298.188652:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188653:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941298.188655:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941298.188656:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941298.188657:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.188658:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941298.188661:0:2395:0:(import.c:2149:obd_at_measure()) add 5 to ffff8897505b1460 time=96 v=5 (5 5 5 5) 00000100:00001000:0.0:1788941298.188663:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b1418 time=140 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941298.188666:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941298.188667:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941298.188668:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941298.188670:0:2395:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.188672:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.188675:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188684:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.188686:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941298.188687:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941298.188691:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941298.188697:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772195500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959590144:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.188701:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188704:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.188707:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772194380 x1875827959590272/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188713:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941298.188715:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941298.188718:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758915800. 02000000:00000001:0.0:1788941298.188722:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941298.188725:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941298.188726:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc8a800 refcount=5 obd=lustre-MDT0000-mdc-ffff889767e18000 00000020:00000001:0.0:1788941298.188729:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941298.188731:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083db00. 02000000:00000001:0.0:1788941298.188735:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941298.188736:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941298.188738:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772194380. 00000100:00000001:0.0:1788941298.188741:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941298.188742:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188743:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.188746:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772195500 x1875827959590144/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941314 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.188751:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941298.188753:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941298.188754:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758915200. 02000000:00000001:0.0:1788941298.188756:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941298.188757:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941298.188761:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b1000 refcount=3 obd=MGC192.168.206.112@tcp 00000020:00000001:0.0:1788941298.188773:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941298.188775:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083d100. 02000000:00000001:0.0:1788941298.188777:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941298.188778:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941298.188780:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772195500. 00000100:00000001:0.0:1788941298.188782:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941298.188783:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188786:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.188792:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.188794:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.188797:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.188799:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788941298.192741:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192760:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941298.192774:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192777:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.192785:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192793:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941298.192802:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192807:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192812:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192816:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 4 MB=0x6aa0e3b30ee80 00000400:00000200:0.0:1788941298.192820:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192826:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.112@tcp of length 224/224 into md 0x537f89 [1] + 224 00000800:00000001:0.0:1788941298.192832:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192836:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.192839:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.192843:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941298.192850:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.192882:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941298.192893:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.192923:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941298.192933:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192937:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941298.192939:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192941:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.192943:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192947:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941298.192952:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192954:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192957:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192960:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30ee00 00000400:00000200:0.0:1788941298.192963:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941298.192968:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x537f99 [1] + 224 00000800:00000001:0.0:1788941298.192971:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.192978:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941298.192980:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941298.192983:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941298.192986:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.192989:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941298.192997:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193003:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941298.193006:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941298.193008:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193021:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.193025:0:2394:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788941298.193030:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941298.193032:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941298.193034:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a170b0. 00000400:00000200:0.0:1788941298.193038:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87f68 00000400:00000010:0.0:1788941298.193046:0:2394:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87f68. 00000100:00000001:0.0:1788941298.193050:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.193055:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941298.193063:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193069:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941298.193071:0:2394:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193087:0:2394:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941298.193090:0:2394:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941298.193092:0:2394:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941298.193094:0:2394:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.193097:0:2394:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941298.193105:0:2394:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc88460 time=96 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941298.193110:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc88418 time=110 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941298.193113:0:2394:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941298.193115:0:2394:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941298.193117:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788941298.193122:0:2394:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788941298.193137:0:2394:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.193141:0:2394:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941298.193144:0:2394:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff889767e18000: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1788941298.193146:0:2394:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941298.193147:0:2394:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.193152:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193159:0:2394:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.193162:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941298.193164:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941298.193167:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941298.193186:0:2394:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772195880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2394:1875827959590400:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941298.193192:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941298.193193:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941298.193195:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17738. 00000400:00000200:0.0:1788941298.193197:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87dd0 00000400:00000010:0.0:1788941298.193198:0:2394:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87dd0. 00000100:00000001:0.0:1788941298.193200:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941298.193203:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941298.193210:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193216:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941298.193217:0:2394:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193219:0:2394:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941298.193220:0:2394:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941298.193221:0:2394:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941298.193223:0:2394:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941298.193224:0:2394:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941298.193228:0:2394:0:(import.c:2149:obd_at_measure()) add 36 to ffff8897505b5c60 time=95 v=36 (36 5 5 5) 00000100:00001000:0.0:1788941298.193231:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b5c18 time=125 v=5 (1 1 1 1) 00000100:00000001:0.0:1788941298.193233:0:2394:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941298.193234:0:2394:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941298.193236:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941298.193238:0:2394:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.193240:0:2394:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941298.193241:0:2394:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff889767e18000: skip recheck: last_committed 42949693494 00000100:00000001:0.0:1788941298.193243:0:2394:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941298.193244:0:2394:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941298.193248:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193254:0:2394:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941298.193262:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941298.193263:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941298.193267:0:2394:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941298.193274:0:2394:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772195180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2394:1875827959590528:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941298.193278:0:2394:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193281:0:2394:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.193284:0:2394:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772195880 x1875827959590400/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941383 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193290:0:2394:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941298.193292:0:2394:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941298.193295:0:2394:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758914400. 02000000:00000001:0.0:1788941298.193298:0:2394:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941298.193300:0:2394:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941298.193302:0:2394:0:(genops.c:1223:class_import_put()) import ffff88976bc88000 refcount=3 obd=lustre-MDT0001-mdc-ffff889767e18000 00000020:00000001:0.0:1788941298.193304:0:2394:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941298.193306:0:2394:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083d900. 02000000:00000001:0.0:1788941298.193309:0:2394:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941298.193311:0:2394:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941298.193312:0:2394:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772195880. 00000100:00000001:0.0:1788941298.193315:0:2394:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941298.193317:0:2394:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193318:0:2394:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941298.193322:0:2394:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772195180 x1875827959590528/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941353 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941298.193327:0:2394:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941298.193328:0:2394:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941298.193330:0:2394:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758914a00. 02000000:00000001:0.0:1788941298.193331:0:2394:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941298.193332:0:2394:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941298.193334:0:2394:0:(genops.c:1223:class_import_put()) import ffff8897505b5800 refcount=3 obd=lustre-OST0000-osc-ffff889767e18000 00000020:00000001:0.0:1788941298.193336:0:2394:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941298.193337:0:2394:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083dd00. 02000000:00000001:0.0:1788941298.193345:0:2394:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941298.193347:0:2394:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941298.193348:0:2394:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772195180. 00000100:00000001:0.0:1788941298.193351:0:2394:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941298.193352:0:2394:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193355:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941298.193362:0:2394:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941298.193363:0:2394:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941298.193365:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941298.193368:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941298.419107:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941298.419113:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941298.419118:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941298.419119:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941298.419149:0:245732:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941298.419152:0:245732:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941298.419155:0:245732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941299.442965:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941299.442971:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941299.442978:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941299.442979:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941299.443032:0:242292:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941299.443035:0:242292:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941299.443039:0:242292:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941300.466985:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941300.466991:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941300.466996:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941300.466997:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0F:1788941300.467085:0:252129:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788941300.467095:0:252129:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788941300.467098:0:252129:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941301.496365:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941301.496374:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941301.496382:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941301.496383:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788941301.496954:0:236344:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788941301.496958:0:236344:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788941301.496962:0:236344:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788941302.003043:0:265097:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af000 (cpu 0) has reached its deadline 4307175137, at 4307175168 00000100:00000010:0.0:1788941302.003058:0:265097:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3840. 00000100:00000040:1.0:1788941302.003389:0:276806:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af110 (cpu 1) has reached its deadline 4307175137, at 4307175168 00000100:00000010:1.0:1788941302.003404:0:276806:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3060. 00010000:00000001:2.0:1788941302.519615:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941302.519622:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941302.519627:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941302.519628:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941302.519648:0:236343:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941302.519651:0:236343:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941302.519653:0:236343:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788941303.282967:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941303.282977:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941303.282988:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196a00. 00000020:00000040:1.0:1788941303.282994:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b1000 refcount=4 obd=MGC192.168.206.112@tcp 00000100:00000001:1.0:1788941303.282999:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941303.283002:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941303.283008:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941303.283016:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083dd00. 00000100:00000001:1.0:1788941303.283023:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283034:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941303.283045:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941303.283048:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283055:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000adde0043] to pc [ptlrpcd_00_01+1] req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.283089:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283092:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941303.283095:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941303.283098:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772197b80. 00000020:00000040:1.0:1788941303.283100:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc8a800 refcount=6 obd=lustre-MDT0000-mdc-ffff889767e18000 00000100:00000001:1.0:1788941303.283103:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941303.283104:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941303.283106:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941303.283109:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083d900. 00000100:00000001:1.0:1788941303.283112:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283116:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941303.283135:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941303.283137:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283141:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f15bcb86] to pc [ptlrpcd_00_00+0] req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.283149:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283151:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941303.283154:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941303.283157:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196300. 00000020:00000040:1.0:1788941303.283159:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc88000 refcount=4 obd=lustre-MDT0001-mdc-ffff889767e18000 00000100:00000001:1.0:1788941303.283161:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941303.283163:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941303.283164:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941303.283168:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083d100. 00000100:00000001:1.0:1788941303.283170:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283174:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941303.283182:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941303.283183:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283187:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e730b484] to pc [ptlrpcd_00_01+1] req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.283194:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283196:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941303.283199:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941303.283202:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772194000. 00000020:00000040:1.0:1788941303.283204:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b5800 refcount=4 obd=lustre-OST0000-osc-ffff889767e18000 00000100:00000001:1.0:1788941303.283206:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941303.283208:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941303.283209:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941303.283212:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976083db00. 00000100:00000001:1.0:1788941303.283220:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283225:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941303.283232:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941303.283234:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283238:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d0d4d31] to pc [ptlrpcd_00_00+0] req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.283244:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.283246:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788941303.283250:0:233298:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788941303.283255:0:233298:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12514) 00000100:00000001:0.0:1788941303.283445:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941303.283449:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941303.283451:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941303.283458:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283465:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941303.283467:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283471:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941303.283473:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941303.283476:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941303.283479:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959590656:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941303.283483:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941303.283487:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941303.283500:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283503:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941303.283508:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758915000. 02000000:00000001:0.0:1788941303.283510:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941303.283516:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941303.283521:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17738. 00000400:00000010:0.0:1788941303.283526:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87dd0. 00000100:00000200:0.0:1788941303.283537:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590656, portal 25 00000100:00000001:0.0:1788941303.283540:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941303.283541:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768548352 : -131290941003264 : ffff889772196a00) 00000100:00000040:0.0:1788941303.283548:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283555:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941303.283558:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941303.283560:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87f68. 00000100:00000200:0.0:1788941303.283563:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875827959590656, offset 0 00000400:00000200:0.0:1788941303.283569:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941303.283577:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.283584:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941303.283588:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941303.283592:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941303.283596:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941303.283599:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941303.283602:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545675:545675:256:4294967295] 192.168.206.112@tcp LPNI seq info [545660:545660:8:4294967295] 00000400:00000200:0.0:1788941303.283607:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.283616:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941303.283622:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941303.283626:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84000. 00000800:00000200:0.0:1788941303.283632:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941303.283638:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941303.283642:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941303.283653:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941303.283658:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941303.283660:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 0 is 4307180255 00000100:00000010:0.0:1788941303.283664:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3840. 00000100:00000001:0.0:1788941303.283670:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788941303.283675:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 1 is 4307180255 00000100:00000010:0.0:1788941303.283678:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3ba0. 00000100:00000001:0.0:1788941303.283685:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788941303.283689:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941303.283691:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.283694:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941303.283698:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283705:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941303.283706:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283708:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941303.283710:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941303.283712:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941303.283714:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590912:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941303.283717:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941303.283720:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941303.283722:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283724:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941303.283726:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758915800. 02000000:00000001:0.0:1788941303.283728:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941303.283731:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941303.283745:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a170b0. 00000400:00000010:0.0:1788941303.283750:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87cc0. 00000100:00000200:0.0:1788941303.283753:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590912, portal 10 00000100:00000001:0.0:1788941303.283755:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941303.283756:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768546560 : -131290941005056 : ffff889772196300) 00000100:00000040:0.0:1788941303.283763:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283769:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941303.283771:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941303.283774:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87908. 00000100:00000200:0.0:1788941303.283776:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959590912, offset 0 00000400:00000200:0.0:1788941303.283779:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941303.283783:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.283794:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941303.283797:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941303.283799:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941303.283802:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941303.283804:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941303.283807:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545676:545676:255:4294967295] 192.168.206.112@tcp LPNI seq info [545661:545661:7:4294967295] 00000400:00000200:0.0:1788941303.283810:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.283817:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941303.283821:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941303.283824:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84200. 00000800:00000200:0.0:1788941303.283828:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941303.283832:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941303.283835:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941303.283838:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941303.283841:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: using a timeout of 20886 usecs (21 jiffies) 00000400:00000200:0.0:1788941303.283844:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941303.283846:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307176469 00000100:00000001:0.0:1788941303.283849:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.283894:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307176469 00000100:00000001:0.0:1788941303.283897:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.283899:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941303.283901:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.283904:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.283906:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941303.283912:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941303.283914:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788941303.283916:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941303.283918:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941303.283919:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941303.283922:0:2395:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:0.0:1788941303.283924:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941303.283931:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941303.283933:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941303.283934:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941303.283935:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941303.283936:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941303.283941:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283948:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941303.283950:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283952:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941303.283953:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941303.283955:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941303.283958:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590784:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941303.283962:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941303.283964:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941303.283965:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.283967:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941303.283970:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889758914a00. 02000000:00000001:0.0:1788941303.283972:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941303.283976:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941303.283978:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17688. 00000400:00000010:0.0:1788941303.283981:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd876e8. 00000100:00000200:0.0:1788941303.283984:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959590784, portal 10 00000100:00000001:0.0:1788941303.283985:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941303.283986:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768552832 : -131290940998784 : ffff889772197b80) 00000100:00000040:0.0:1788941303.283992:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.283998:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941303.284000:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941303.284002:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87000. 00000100:00000200:0.0:1788941303.284004:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959590784, offset 0 00000400:00000200:0.0:1788941303.284009:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941303.284013:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.284022:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941303.284025:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941303.284027:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941303.284030:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941303.284032:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941303.284035:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545677:545677:254:4294967295] 192.168.206.112@tcp LPNI seq info [545662:545662:6:4294967295] 00000400:00000200:0.0:1788941303.284039:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.284045:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941303.284049:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941303.284053:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84500. 00000800:00000200:0.0:1788941303.284057:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941303.284061:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941303.284063:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941303.284066:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941303.284069:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: using a timeout of 34617 usecs (35 jiffies) 00000400:00000200:0.0:1788941303.284072:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941303.284074:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307176483 00000100:00000001:0.0:1788941303.284076:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.284078:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307176483 00000100:00000001:0.0:1788941303.284080:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.284082:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941303.284084:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284086:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941303.284090:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284096:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941303.284098:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.284100:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941303.284102:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941303.284109:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941303.284113:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591040:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941303.284116:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941303.284118:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941303.284120:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941303.284122:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941303.284125:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8897516e0a00. 02000000:00000001:0.0:1788941303.284127:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941303.284130:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941303.284133:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a172c0. 00000400:00000010:0.0:1788941303.284136:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87e58. 00000100:00000200:0.0:1788941303.284139:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591040, portal 4 00000100:00000001:0.0:1788941303.284140:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941303.284142:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768537600 : -131290941014016 : ffff889772194000) 00000100:00000040:0.0:1788941303.284147:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284153:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941303.284154:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941303.284156:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87990. 00000100:00000200:0.0:1788941303.284159:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875827959591040, offset 0 00000400:00000200:0.0:1788941303.284162:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941303.284166:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.284169:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941303.284171:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941303.284173:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941303.284175:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941303.284177:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941303.284180:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545678:545678:253:4294967295] 192.168.206.112@tcp LPNI seq info [545663:545663:5:4294967295] 00000400:00000200:0.0:1788941303.284183:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.284188:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941303.284192:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941303.284200:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84100. 00000800:00000200:0.0:1788941303.284204:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941303.284208:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941303.284210:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941303.284213:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941303.284216:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: using a timeout of 139340 usecs (140 jiffies) 00000400:00000200:0.0:1788941303.284219:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941303.284221:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307176588 00000100:00000001:0.0:1788941303.284223:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.284225:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307176588 00000100:00000001:0.0:1788941303.284227:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307180255 00000100:00000001:0.0:1788941303.284229:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941303.284231:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284233:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284235:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284245:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941303.284248:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788941303.284299:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941303.284302:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84000. 00000400:00000200:0.0:1788941303.284308:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.284312:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941303.284317:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87f68 00000400:00000010:0.0:1788941303.284319:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87f68. 00000100:00000001:0.0:1788941303.284323:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941303.284327:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284337:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941303.284340:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284347:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284349:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941303.284368:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941303.284371:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84200. 00000400:00000200:0.0:1788941303.284373:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.284377:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941303.284379:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87908 00000400:00000010:0.0:1788941303.284381:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87908. 00000100:00000001:0.0:1788941303.284383:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941303.284387:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284394:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941303.284396:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284402:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284404:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941303.284411:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941303.284413:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84500. 00000400:00000200:0.0:1788941303.284415:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.284418:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941303.284420:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87000 00000400:00000010:0.0:1788941303.284422:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87000. 00000100:00000001:0.0:1788941303.284424:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941303.284427:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284434:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941303.284437:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284443:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284444:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941303.284499:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941303.284502:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84100. 00000400:00000200:0.0:1788941303.284505:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.284508:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941303.284510:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87990 00000400:00000010:0.0:1788941303.284517:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87990. 00000100:00000001:0.0:1788941303.284520:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941303.284524:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284532:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941303.284535:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.284542:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941303.284544:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788941303.292820:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.292835:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941303.292839:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.292841:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.292849:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.292881:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941303.292890:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.292895:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.292899:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.292903:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 4 MB=0x6aa0e3b30f080 00000400:00000200:0.0:1788941303.292907:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.292913:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.112@tcp of length 224/224 into md 0x537fd9 [1] + 224 00000800:00000001:0.0:1788941303.292920:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.292924:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.292927:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.292931:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941303.292937:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941303.292945:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941303.292956:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.293029:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.293035:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941303.293054:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941303.293178:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.293182:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941303.293184:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.293186:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.293190:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.293194:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941303.293199:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.293201:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.293204:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.293208:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30f000 00000400:00000200:0.0:1788941303.293211:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.293216:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x537fb9 [1] + 224 00000800:00000001:0.0:1788941303.293220:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.293226:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.293228:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.293231:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941303.293235:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941303.293239:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941303.293248:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.293256:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941303.293259:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.293262:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000001:1.0:1788941303.295698:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788941303.295705:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788941303.295709:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a172c0. 00000400:00000200:1.0:1788941303.295717:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87e58 00000400:00000010:1.0:1788941303.295719:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87e58. 00000100:00000001:1.0:1788941303.295723:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788941303.295733:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788941303.295755:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.295764:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788941303.295766:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.295769:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788941303.295772:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788941303.295774:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788941303.295786:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788941303.295789:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788941303.295798:0:2395:0:(import.c:2149:obd_at_measure()) add 36 to ffff8897505b5c60 time=100 v=36 (36 5 5 5) 00000100:00001000:1.0:1788941303.295803:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b5c18 time=130 v=5 (1 1 1 1) 00000100:00000001:1.0:1788941303.295820:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788941303.295822:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788941303.295825:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788941303.295828:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.295831:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788941303.295834:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff889767e18000: skip recheck: last_committed 42949693494 00000100:00000001:1.0:1788941303.295836:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788941303.295837:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.295842:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.295850:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788941303.295867:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788941303.295869:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788941303.295873:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788941303.295881:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591040:192.168.206.112@tcp:400:kworker.0 00000400:00000001:1.0:1788941303.295889:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788941303.295891:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788941303.295893:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a170b0. 00000400:00000200:1.0:1788941303.295896:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87cc0 00000400:00000010:1.0:1788941303.295898:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87cc0. 00000100:00000001:1.0:1788941303.295900:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788941303.295904:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788941303.295913:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.295919:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788941303.295920:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.295922:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788941303.295923:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788941303.295925:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788941303.295926:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788941303.295934:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788941303.295939:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc88460 time=101 v=60 (60 60 60 61) 00000100:00001000:1.0:1788941303.295942:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc88418 time=115 v=5 (1 1 0 0) 00000100:00000001:1.0:1788941303.295945:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788941303.295947:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788941303.295949:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1788941303.295953:0:2395:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1788941303.295960:0:2395:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.295962:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788941303.295964:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff889767e18000: skip recheck: last_committed 21474836545 00000100:00000001:1.0:1788941303.295965:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788941303.295967:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.295970:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.295976:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788941303.295978:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788941303.295980:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788941303.295983:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788941303.295990:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590912:192.168.206.112@tcp:400:kworker.0 00000100:00000001:1.0:1788941303.295994:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.295997:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941303.296000:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772194000 x1875827959591040/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941358 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.296007:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788941303.296009:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788941303.296012:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8897516e0a00. 02000000:00000001:1.0:1788941303.296016:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788941303.296018:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788941303.296020:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b5800 refcount=3 obd=lustre-OST0000-osc-ffff889767e18000 00000020:00000001:1.0:1788941303.296028:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788941303.296031:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083db00. 02000000:00000001:1.0:1788941303.296034:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788941303.296035:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788941303.296037:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772194000. 00000100:00000001:1.0:1788941303.296040:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788941303.296042:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.296044:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941303.296048:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196300 x1875827959590912/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.296054:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788941303.296055:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788941303.296164:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758915800. 02000000:00000001:1.0:1788941303.296170:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788941303.296171:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788941303.296173:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc88000 refcount=3 obd=lustre-MDT0001-mdc-ffff889767e18000 00000020:00000001:1.0:1788941303.296174:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788941303.296176:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083d100. 02000000:00000001:1.0:1788941303.296178:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788941303.296179:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788941303.296181:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196300. 00000100:00000001:1.0:1788941303.296183:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788941303.296184:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.296187:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.296189:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.296191:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788941303.296193:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.296195:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788941303.299053:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.299081:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941303.299085:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.299087:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.299095:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.299103:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941303.299113:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.299117:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.299122:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.299137:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 25 MB=0x6aa0e3b30ef00 00000400:00000200:0.0:1788941303.299142:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.299147:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.112@tcp of length 224/224 into md 0x537fa9 [1] + 224 00000800:00000001:0.0:1788941303.299154:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.299161:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.299164:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.299168:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941303.299175:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941303.299183:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941303.299195:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.299217:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941303.299225:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.299228:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.299350:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.299354:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788941303.299361:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788941303.299363:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788941303.299366:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17738. 00000400:00000200:1.0:1788941303.299371:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87dd0 00000400:00000010:1.0:1788941303.299374:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87dd0. 00000100:00000001:1.0:1788941303.299376:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788941303.299381:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788941303.299402:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.299408:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788941303.299410:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.299412:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788941303.299416:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788941303.299431:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788941303.299434:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788941303.299436:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788941303.299442:0:2395:0:(import.c:2149:obd_at_measure()) add 5 to ffff8897505b1460 time=101 v=5 (5 5 5 5) 00000100:00001000:1.0:1788941303.299447:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b1418 time=145 v=5 (1 1 0 0) 00000100:00000001:1.0:1788941303.299449:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788941303.299451:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788941303.299454:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788941303.299455:0:2395:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.299459:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.299463:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.299470:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788941303.299473:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788941303.299475:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788941303.299479:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788941303.299486:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959590656:192.168.206.112@tcp:400:kworker.0 00000100:00000001:1.0:1788941303.299491:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.299494:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941303.299497:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196a00 x1875827959590656/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941319 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.299503:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788941303.299506:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788941303.299515:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758915000. 02000000:00000001:1.0:1788941303.299519:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788941303.299522:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788941303.299524:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b1000 refcount=3 obd=MGC192.168.206.112@tcp 00000020:00000001:1.0:1788941303.299526:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788941303.299528:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083dd00. 02000000:00000001:1.0:1788941303.299531:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788941303.299533:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788941303.299535:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196a00. 00000100:00000001:1.0:1788941303.299537:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788941303.299538:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.299541:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.299543:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.299544:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1788941303.299546:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.299548:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788941303.304909:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.304925:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941303.304929:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.304932:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.304940:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.304948:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941303.304957:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.304962:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.304966:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.304970:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30ef80 00000400:00000200:0.0:1788941303.304975:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941303.304981:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x537fc9 [1] + 224 00000800:00000001:0.0:1788941303.304988:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.304995:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941303.304999:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941303.305003:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941303.305010:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941303.305032:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941303.305043:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941303.305077:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941303.305087:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941303.305090:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.305212:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.305217:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1788941303.305222:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1788941303.305224:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1788941303.305227:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17688. 00000400:00000200:1.0:1788941303.305231:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd876e8 00000400:00000010:1.0:1788941303.305233:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd876e8. 00000100:00000001:1.0:1788941303.305236:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1788941303.305241:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1788941303.305249:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.305255:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1788941303.305257:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.305260:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:1.0:1788941303.305263:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1788941303.305265:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1788941303.305267:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1788941303.305270:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1788941303.305279:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc8ac60 time=101 v=60 (60 60 60 61) 00000100:00001000:1.0:1788941303.305283:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc8ac18 time=145 v=5 (1 1 0 0) 00000100:00000001:1.0:1788941303.305292:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1788941303.305295:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1788941303.305298:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1788941303.305301:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.305304:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1788941303.305306:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff889767e18000: skip recheck: last_committed 30064771462 00000100:00000001:1.0:1788941303.305308:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1788941303.305310:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941303.305316:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.305323:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1788941303.305326:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1788941303.305328:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1788941303.305331:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1788941303.305339:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959590784:192.168.206.112@tcp:400:kworker.0 00000100:00000001:1.0:1788941303.305344:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.305347:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941303.305350:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772197b80 x1875827959590784/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941388 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941303.305356:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1788941303.305358:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1788941303.305362:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889758914a00. 02000000:00000001:1.0:1788941303.305366:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1788941303.305368:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:1.0:1788941303.305370:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc8a800 refcount=5 obd=lustre-MDT0000-mdc-ffff889767e18000 00000020:00000001:1.0:1788941303.305372:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:1.0:1788941303.305374:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976083d900. 02000000:00000001:1.0:1788941303.305378:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1788941303.305379:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1788941303.305381:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772197b80. 00000100:00000001:1.0:1788941303.305384:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1788941303.305391:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.305394:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1788941303.305401:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1788941303.305403:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941303.305405:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1788941303.305407:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941303.538953:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941303.538959:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941303.538964:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941303.538981:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941303.539030:0:245732:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941303.539033:0:245732:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941303.539036:0:245732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941304.563009:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941304.563017:0:250116:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788941304.563021:0:250116:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788941304.563023:0:250116:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563028:0:250116:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788941304.563030:0:250116:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563032:0:250116:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563034:0:250116:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563037:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941304.563039:0:250116:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788941304.563041:0:250116:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788941304.563045:0:250116:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788941304.563047:0:250116:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788941304.563048:0:250116:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563049:0:250116:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788941304.563051:0:250116:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563052:0:250116:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941304.563054:0:250116:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941304.563072:0:242292:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941304.563075:0:242292:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941304.563077:0:242292:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941305.596551:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941305.596559:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941305.596565:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941305.596567:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788941305.598376:0:252129:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788941305.598384:0:252129:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788941305.598389:0:252129:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941306.611010:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941306.611019:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941306.611026:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941306.611028:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788941306.611240:0:236344:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788941306.611244:0:236344:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788941306.611248:0:236344:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1788941307.123924:0:265097:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af000 (cpu 0) has reached its deadline 4307180255, at 4307180288 00000100:00000010:0.0:1788941307.123937:0:265097:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3840. 00000100:00000040:1.0:1788941307.123993:0:276806:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af110 (cpu 1) has reached its deadline 4307180255, at 4307180288 00000100:00000010:1.0:1788941307.123999:0:276806:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3ba0. 00000080:00000001:0.0:1788941307.135706:0:278701:0:(llite_lib.c:4247:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941307.636004:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941307.636011:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941307.636016:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941307.636018:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941307.636035:0:236343:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941307.636051:0:236343:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941307.636054:0:236343:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788941308.403024:0:233979:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941308.403038:0:233979:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941308.403054:0:233979:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772197b80. 00000020:00000040:1.0:1788941308.403061:0:233979:0:(genops.c:1210:class_import_get()) import ffff8897505b1000 refcount=4 obd=MGC192.168.206.112@tcp 00000100:00000001:1.0:1788941308.403068:0:233979:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941308.403072:0:233979:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941308.403079:0:233979:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941308.403089:0:233979:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5000. 00000100:00000001:1.0:1788941308.403098:0:233979:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403115:0:233979:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941308.403128:0:233979:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941308.403133:0:233979:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403140:0:233979:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f15bcb86] to pc [ptlrpcd_00_01+1] req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.403179:0:233979:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403183:0:233979:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941308.403187:0:233979:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941308.403190:0:233979:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196a00. 00000020:00000040:1.0:1788941308.403193:0:233979:0:(genops.c:1210:class_import_get()) import ffff88976bc8a800 refcount=6 obd=lustre-MDT0000-mdc-ffff889767e18000 00000100:00000001:1.0:1788941308.403195:0:233979:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941308.403196:0:233979:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941308.403198:0:233979:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941308.403202:0:233979:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5f00. 00000100:00000001:1.0:1788941308.403206:0:233979:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403230:0:233979:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941308.403237:0:233979:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941308.403239:0:233979:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403243:0:233979:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000adde0043] to pc [ptlrpcd_00_00+0] req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.403252:0:233979:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403254:0:233979:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941308.403258:0:233979:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941308.403261:0:233979:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196300. 00000020:00000040:1.0:1788941308.403264:0:233979:0:(genops.c:1210:class_import_get()) import ffff88976bc88000 refcount=4 obd=lustre-MDT0001-mdc-ffff889767e18000 00000100:00000001:1.0:1788941308.403266:0:233979:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941308.403268:0:233979:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941308.403270:0:233979:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941308.403273:0:233979:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5200. 00000100:00000001:1.0:1788941308.403277:0:233979:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403281:0:233979:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941308.403288:0:233979:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941308.403290:0:233979:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403293:0:233979:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e730b484] to pc [ptlrpcd_00_01+1] req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.403301:0:233979:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403303:0:233979:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941308.403307:0:233979:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941308.403310:0:233979:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772194000. 00000020:00000040:1.0:1788941308.403313:0:233979:0:(genops.c:1210:class_import_get()) import ffff8897505b5800 refcount=4 obd=lustre-OST0000-osc-ffff889767e18000 00000100:00000001:1.0:1788941308.403315:0:233979:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941308.403316:0:233979:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941308.403318:0:233979:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941308.403327:0:233979:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5100. 00000100:00000001:1.0:1788941308.403330:0:233979:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403334:0:233979:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941308.403341:0:233979:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941308.403343:0:233979:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403346:0:233979:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d0d4d31] to pc [ptlrpcd_00_00+0] req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.403353:0:233979:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941308.403355:0:233979:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788941308.403358:0:233979:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788941308.403364:0:233979:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12519) 00000100:00000001:0.0:1788941308.403649:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.403655:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941308.403658:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941308.403666:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.403675:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941308.403678:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.403683:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941308.403686:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941308.403689:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941308.403692:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959591168:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941308.403697:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941308.403704:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941308.403716:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.403720:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941308.403725:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748472200. 02000000:00000001:0.0:1788941308.403728:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941308.403736:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941308.403741:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17108. 00000400:00000010:0.0:1788941308.403756:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87990. 00000100:00000200:0.0:1788941308.403764:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591168, portal 25 00000100:00000001:0.0:1788941308.403767:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941308.403769:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768552832 : -131290940998784 : ffff889772197b80) 00000100:00000040:0.0:1788941308.403775:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.403783:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941308.403786:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941308.403789:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87000. 00000100:00000200:0.0:1788941308.403792:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875827959591168, offset 0 00000400:00000200:0.0:1788941308.403800:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941308.403811:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.403820:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941308.403826:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941308.403831:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941308.403835:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941308.403838:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941308.403842:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545679:545679:256:4294967295] 192.168.206.112@tcp LPNI seq info [545664:545664:8:4294967295] 00000400:00000200:0.0:1788941308.403846:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.403907:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941308.403914:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941308.403920:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84100. 00000800:00000200:0.0:1788941308.403927:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941308.403938:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941308.403944:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84100 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941308.403954:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941308.403959:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941308.403962:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 0 is 4307185376 00000100:00000010:0.0:1788941308.403966:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d35a0. 00000100:00000001:0.0:1788941308.403974:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788941308.403981:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 1 is 4307185376 00000100:00000010:0.0:1788941308.403990:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3000. 00000100:00000001:0.0:1788941308.403994:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788941308.403998:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.404001:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404004:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941308.404009:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404015:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941308.404017:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404020:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941308.404021:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941308.404023:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941308.404025:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591424:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941308.404029:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941308.404032:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941308.404033:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404035:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941308.404038:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748473e00. 02000000:00000001:0.0:1788941308.404040:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941308.404043:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941308.404045:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17f20. 00000400:00000010:0.0:1788941308.404050:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87908. 00000100:00000200:0.0:1788941308.404052:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591424, portal 10 00000100:00000001:0.0:1788941308.404054:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941308.404055:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768546560 : -131290941005056 : ffff889772196300) 00000100:00000040:0.0:1788941308.404062:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404067:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941308.404069:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941308.404071:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87f68. 00000100:00000200:0.0:1788941308.404074:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959591424, offset 0 00000400:00000200:0.0:1788941308.404078:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404088:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404092:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941308.404094:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941308.404097:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941308.404099:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941308.404101:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404104:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545680:545680:255:4294967295] 192.168.206.112@tcp LPNI seq info [545665:545665:7:4294967295] 00000400:00000200:0.0:1788941308.404107:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404112:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941308.404116:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941308.404120:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84000. 00000800:00000200:0.0:1788941308.404123:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941308.404128:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941308.404130:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84000 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941308.404133:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.404137:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: using a timeout of 20886 usecs (21 jiffies) 00000400:00000200:0.0:1788941308.404139:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941308.404141:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307181590 00000100:00000001:0.0:1788941308.404143:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404145:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307181590 00000100:00000001:0.0:1788941308.404147:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404149:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.404151:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404154:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404157:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.404165:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941308.404168:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788941308.404170:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.404172:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941308.404173:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941308.404176:0:2395:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:0.0:1788941308.404182:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.404185:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941308.404186:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.404188:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.404189:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941308.404190:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941308.404195:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404202:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941308.404203:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404206:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941308.404207:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941308.404209:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941308.404213:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591296:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941308.404216:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941308.404219:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941308.404220:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404222:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941308.404226:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748473200. 02000000:00000001:0.0:1788941308.404228:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941308.404232:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941308.404235:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17cb8. 00000400:00000010:0.0:1788941308.404238:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87880. 00000100:00000200:0.0:1788941308.404241:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591296, portal 10 00000100:00000001:0.0:1788941308.404243:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941308.404244:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768548352 : -131290941003264 : ffff889772196a00) 00000100:00000040:0.0:1788941308.404250:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404255:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941308.404258:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941308.404260:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87198. 00000100:00000200:0.0:1788941308.404262:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959591296, offset 0 00000400:00000200:0.0:1788941308.404266:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404270:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404279:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941308.404282:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941308.404284:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941308.404286:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941308.404289:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404291:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545681:545681:254:4294967295] 192.168.206.112@tcp LPNI seq info [545666:545666:6:4294967295] 00000400:00000200:0.0:1788941308.404295:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404300:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941308.404304:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941308.404307:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84c00. 00000800:00000200:0.0:1788941308.404311:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941308.404315:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941308.404317:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84c00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941308.404321:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.404324:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: using a timeout of 34616 usecs (35 jiffies) 00000400:00000200:0.0:1788941308.404327:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941308.404329:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307181604 00000100:00000001:0.0:1788941308.404331:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404332:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307181604 00000100:00000001:0.0:1788941308.404334:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404336:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.404338:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404340:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941308.404344:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404350:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941308.404352:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404354:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941308.404355:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941308.404362:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941308.404366:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591552:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941308.404369:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941308.404371:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941308.404373:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.404375:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941308.404378:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889743983c00. 02000000:00000001:0.0:1788941308.404380:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941308.404384:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941308.404387:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17370. 00000400:00000010:0.0:1788941308.404403:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87330. 00000100:00000200:0.0:1788941308.404407:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591552, portal 4 00000100:00000001:0.0:1788941308.404409:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941308.404410:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768537600 : -131290941014016 : ffff889772194000) 00000100:00000040:0.0:1788941308.404417:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.404424:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941308.404426:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941308.404428:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87b28. 00000100:00000200:0.0:1788941308.404430:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875827959591552, offset 0 00000400:00000200:0.0:1788941308.404434:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404438:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404441:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941308.404444:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941308.404446:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941308.404448:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941308.404451:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941308.404453:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545682:545682:253:4294967295] 192.168.206.112@tcp LPNI seq info [545667:545667:5:4294967295] 00000400:00000200:0.0:1788941308.404457:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.404462:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941308.404466:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941308.404475:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84e00. 00000800:00000200:0.0:1788941308.404479:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941308.404484:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941308.404486:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84e00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941308.404489:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.404492:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: using a timeout of 139248 usecs (140 jiffies) 00000400:00000200:0.0:1788941308.404495:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941308.404496:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307181709 00000100:00000001:0.0:1788941308.404499:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404500:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307181709 00000100:00000001:0.0:1788941308.404502:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307185376 00000100:00000001:0.0:1788941308.404504:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.404506:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404509:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.404511:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1788941308.405267:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941308.405275:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84100. 00000400:00000200:1.0:1788941308.405285:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941308.405292:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941308.405299:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87000 00000400:00000010:1.0:1788941308.405302:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87000. 00000100:00000001:1.0:1788941308.405307:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941308.405314:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405325:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941308.405328:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405335:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941308.405337:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788941308.405352:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941308.405354:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84000. 00000400:00000200:1.0:1788941308.405357:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941308.405361:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941308.405364:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87f68 00000400:00000010:1.0:1788941308.405375:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87f68. 00000100:00000001:1.0:1788941308.405378:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941308.405382:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405401:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941308.405404:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405409:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941308.405410:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788941308.405421:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941308.405423:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84c00. 00000400:00000200:1.0:1788941308.405426:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941308.405429:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941308.405431:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87198 00000400:00000010:1.0:1788941308.405433:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87198. 00000100:00000001:1.0:1788941308.405435:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941308.405439:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405445:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941308.405448:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405453:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941308.405454:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1788941308.405530:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1788941308.405534:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84e00. 00000400:00000200:1.0:1788941308.405537:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1788941308.405540:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:1.0:1788941308.405543:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87b28 00000400:00000010:1.0:1788941308.405545:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87b28. 00000100:00000001:1.0:1788941308.405547:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1788941308.405551:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405573:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1788941308.405576:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941308.405582:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1788941308.405583:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1788941308.423189:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.423202:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1788941308.423228:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.423251:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941308.423254:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.423257:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.423267:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.423277:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941308.423290:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.423296:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.423302:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.423307:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30f200 00000400:00000200:0.0:1788941308.423312:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.423319:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x537ff9 [1] + 224 00000800:00000001:0.0:1788941308.423328:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.423333:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.423337:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941308.423361:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941308.423371:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.423383:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941308.427956:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.427986:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941308.428009:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428029:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941308.428032:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428034:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428042:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428048:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941308.428056:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428061:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428066:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428070:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30f180 00000400:00000200:0.0:1788941308.428075:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428081:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x538009 [1] + 224 00000800:00000001:0.0:1788941308.428089:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428093:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428096:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941308.428101:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941308.428107:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.428112:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941308.428120:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.428128:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941308.428132:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428149:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941308.428151:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428153:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428157:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428160:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941308.428164:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428167:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428170:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428172:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 4 MB=0x6aa0e3b30f280 00000400:00000200:0.0:1788941308.428176:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428180:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.112@tcp of length 224/224 into md 0x538019 [1] + 224 00000800:00000001:0.0:1788941308.428184:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428187:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428189:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941308.428192:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941308.428195:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.428199:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941308.428207:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.428213:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941308.428216:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428219:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941308.428221:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428222:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428225:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428227:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941308.428231:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428234:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428236:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428239:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 25 MB=0x6aa0e3b30f100 00000400:00000200:0.0:1788941308.428243:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941308.428253:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.112@tcp of length 224/224 into md 0x537fe9 [1] + 224 00000800:00000001:0.0:1788941308.428258:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428278:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941308.428281:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941308.428285:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941308.428288:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.428293:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941308.428302:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.428309:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941308.428313:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941308.428316:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.428342:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.428346:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788941308.428354:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941308.428356:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941308.428360:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17cb8. 00000400:00000200:0.0:1788941308.428364:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87880 00000400:00000010:0.0:1788941308.428367:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87880. 00000100:00000001:0.0:1788941308.428370:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.428375:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941308.428383:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.428389:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941308.430744:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.430752:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941308.430756:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941308.430759:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941308.430763:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.430766:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941308.430779:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc8ac60 time=106 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941308.430798:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc8ac18 time=150 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941308.430801:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941308.430803:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941308.430806:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941308.430811:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.430816:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941308.430818:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff889767e18000: skip recheck: last_committed 30064771462 00000100:00000001:0.0:1788941308.430821:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941308.430823:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.430836:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.430844:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.430847:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941308.430850:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941308.430871:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941308.430879:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591296:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941308.430990:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941308.430993:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941308.430998:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17370. 00000400:00000200:0.0:1788941308.431003:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87330 00000400:00000010:0.0:1788941308.431006:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87330. 00000100:00000001:0.0:1788941308.431010:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.431015:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941308.431025:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431031:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941308.431032:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431034:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941308.431036:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941308.431044:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941308.431046:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.431048:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941308.431054:0:2395:0:(import.c:2149:obd_at_measure()) add 36 to ffff8897505b5c60 time=105 v=36 (36 5 5 5) 00000100:00001000:0.0:1788941308.431057:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b5c18 time=135 v=5 (1 1 1 1) 00000100:00000001:0.0:1788941308.431059:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941308.431060:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941308.431062:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941308.431064:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.431066:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941308.431067:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff889767e18000: skip recheck: last_committed 42949693494 00000100:00000001:0.0:1788941308.431069:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941308.431070:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.431074:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431080:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.431081:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941308.431083:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941308.431086:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941308.431092:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591552:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941308.431098:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941308.431100:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941308.431102:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17108. 00000400:00000200:0.0:1788941308.431105:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87990 00000400:00000010:0.0:1788941308.431107:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87990. 00000100:00000001:0.0:1788941308.431109:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.431113:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941308.431120:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431126:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941308.431134:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431136:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941308.431137:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941308.431138:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941308.431140:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.431142:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941308.431145:0:2395:0:(import.c:2149:obd_at_measure()) add 5 to ffff8897505b1460 time=106 v=5 (5 5 5 5) 00000100:00001000:0.0:1788941308.431148:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b1418 time=150 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941308.431150:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941308.431151:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941308.431153:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941308.431155:0:2395:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.431156:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.431160:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431166:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.431168:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941308.431170:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941308.431173:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941308.431180:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959591168:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941308.431186:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941308.431187:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941308.431189:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17f20. 00000400:00000200:0.0:1788941308.431191:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87908 00000400:00000010:0.0:1788941308.431193:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87908. 00000100:00000001:0.0:1788941308.431195:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941308.431199:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941308.431207:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431213:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941308.431220:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431222:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941308.431223:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941308.431224:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941308.431226:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941308.431228:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941308.431233:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc88460 time=106 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941308.431236:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc88418 time=120 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941308.431238:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941308.431240:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941308.431241:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788941308.431246:0:2395:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788941308.431252:0:2395:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.431254:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941308.431256:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff889767e18000: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1788941308.431257:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941308.431258:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941308.431261:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431267:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941308.431268:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941308.431270:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941308.431273:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941308.431279:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591424:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941308.431283:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431287:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941308.431290:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196a00 x1875827959591296/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431296:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941308.431305:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941308.431309:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748473200. 02000000:00000001:0.0:1788941308.431314:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941308.431317:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941308.431319:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc8a800 refcount=5 obd=lustre-MDT0000-mdc-ffff889767e18000 00000020:00000001:0.0:1788941308.431322:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941308.431325:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5f00. 02000000:00000001:0.0:1788941308.431329:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941308.431331:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941308.431334:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196a00. 00000100:00000001:0.0:1788941308.431338:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941308.431339:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431341:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941308.431345:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772194000 x1875827959591552/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941363 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431351:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941308.431352:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941308.431353:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889743983c00. 02000000:00000001:0.0:1788941308.431361:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941308.431362:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941308.431363:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b5800 refcount=3 obd=lustre-OST0000-osc-ffff889767e18000 00000020:00000001:0.0:1788941308.431365:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941308.431367:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5100. 02000000:00000001:0.0:1788941308.431369:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941308.431370:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941308.431371:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772194000. 00000100:00000001:0.0:1788941308.431373:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941308.431374:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431376:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941308.431379:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772197b80 x1875827959591168/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941324 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431384:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941308.431386:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941308.431387:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748472200. 02000000:00000001:0.0:1788941308.431390:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941308.431391:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941308.431398:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b1000 refcount=3 obd=MGC192.168.206.112@tcp 00000020:00000001:0.0:1788941308.431400:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941308.431402:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5000. 02000000:00000001:0.0:1788941308.431404:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941308.431405:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941308.431407:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772197b80. 00000100:00000001:0.0:1788941308.431409:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941308.431410:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431412:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941308.431415:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196300 x1875827959591424/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941393 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941308.431421:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941308.431422:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941308.431423:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748473e00. 02000000:00000001:0.0:1788941308.431426:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941308.431427:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941308.431428:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc88000 refcount=3 obd=lustre-MDT0001-mdc-ffff889767e18000 00000020:00000001:0.0:1788941308.431430:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941308.431431:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5200. 02000000:00000001:0.0:1788941308.431432:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941308.431433:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941308.431434:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196300. 00000100:00000001:0.0:1788941308.431436:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941308.431437:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431440:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941308.431450:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941308.431452:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941308.431455:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941308.431458:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941308.659569:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941308.659578:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941308.659585:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941308.659586:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941308.659609:0:245732:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941308.659612:0:245732:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941308.659615:0:245732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000200:3.0:1788941309.427201:0:2385:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889742a84480] -> 12345-192.168.206.12@tcp (6) 00000800:00000200:3.0:1788941309.427211:0:2385:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.12@tcp ip 192.168.206.12:988 00000800:00000200:3.0:1788941309.427217:0:2385:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff889751caa400 type 192, nob 24 niov 1 nkiov 0 00000800:00000200:1.0:1788941309.434941:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000001:1.0:1788941309.434954:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788941309.434968:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1788941309.434973:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1788941309.434976:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:2.0:1788941309.684367:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941309.684376:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941309.684632:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941309.684634:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941309.684712:0:242292:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941309.684716:0:242292:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941309.684719:0:242292:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941310.707065:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941310.707075:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941310.707083:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941310.707085:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788941310.707192:0:252129:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788941310.707196:0:252129:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788941310.707199:0:252129:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941311.731774:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941311.731780:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941311.731785:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941311.731786:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1788941311.732680:0:236344:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1788941311.732684:0:236344:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1788941311.732687:0:236344:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788941312.243120:0:276806:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af110 (cpu 1) has reached its deadline 4307185376, at 4307185408 00000100:00000010:1.0:1788941312.243134:0:276806:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3000. 00000100:00000040:0.0:1788941312.243741:0:265097:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af000 (cpu 0) has reached its deadline 4307185376, at 4307185408 00000100:00000010:0.0:1788941312.243748:0:265097:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d35a0. 00010000:00000001:2.0:1788941312.755676:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941312.755682:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941312.755699:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941312.755700:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788941312.756719:0:236343:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788941312.756724:0:236343:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788941312.756727:0:236343:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1788941313.526095:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941313.526103:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941313.526109:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196300. 00000020:00000040:1.0:1788941313.526115:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b1000 refcount=4 obd=MGC192.168.206.112@tcp 00000100:00000001:1.0:1788941313.526119:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941313.526122:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941313.526127:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941313.526133:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5b00. 00000100:00000001:1.0:1788941313.526140:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526151:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 3b28a452-9a8c-4ca1-ada0-d01d4e7186fd->MGS req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941313.526162:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941313.526165:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526172:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e730b484] to pc [ptlrpcd_00_01+1] req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941313.526189:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526191:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941313.526196:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941313.526207:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772197b80. 00000020:00000040:1.0:1788941313.526210:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc8a800 refcount=6 obd=lustre-MDT0000-mdc-ffff889767e18000 00000100:00000001:1.0:1788941313.526212:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941313.526214:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941313.526216:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941313.526221:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5500. 00000100:00000001:1.0:1788941313.526224:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526229:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0000_UUID req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941313.526236:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941313.526238:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526241:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f15bcb86] to pc [ptlrpcd_00_00+0] req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941313.526251:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526254:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941313.526257:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941313.526260:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772194000. 00000020:00000040:1.0:1788941313.526263:0:233298:0:(genops.c:1210:class_import_get()) import ffff88976bc88000 refcount=4 obd=lustre-MDT0001-mdc-ffff889767e18000 00000100:00000001:1.0:1788941313.526266:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941313.526267:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941313.526269:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941313.526274:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5e00. 00000100:00000001:1.0:1788941313.526277:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526283:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-MDT0001_UUID req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941313.526289:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941313.526292:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526296:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000006d0d4d31] to pc [ptlrpcd_00_01+1] req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941313.526304:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526312:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1788941313.526315:0:233298:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1788941313.526324:0:233298:0:(client.c:606:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff889772196a00. 00000020:00000040:1.0:1788941313.526328:0:233298:0:(genops.c:1210:class_import_get()) import ffff8897505b5800 refcount=4 obd=lustre-OST0000-osc-ffff889767e18000 00000100:00000001:1.0:1788941313.526330:0:233298:0:(client.c:858:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1788941313.526332:0:233298:0:(sec.c:433:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1788941313.526334:0:233298:0:(sec.c:457:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1788941313.526338:0:233298:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff88976d0d5700. 00000100:00000001:1.0:1788941313.526340:0:233298:0:(client.c:938:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526345:0:233298:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0000_UUID req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1788941313.526351:0:233298:0:(jobid.c:939:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1788941313.526353:0:233298:0:(jobid.c:979:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526357:0:233298:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000adde0043] to pc [ptlrpcd_00_00+0] req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1788941313.526363:0:233298:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941313.526365:0:233298:0:(pinger.c:229:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1788941313.526367:0:233298:0:(pinger.c:247:ptlrpc_pinger_process_import()) 5577a8ea-8937-4a4d-9ce6-14bd397bfba4->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1788941313.526372:0:233298:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12524) 00000100:00000001:0.0:1788941313.526874:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.526879:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941313.526888:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941313.526894:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.526902:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941313.526905:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.526908:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941313.526912:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941313.526914:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941313.526918:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959591680:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941313.526921:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941313.526927:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941313.526929:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.526931:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941313.526935:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748472200. 02000000:00000001:0.0:1788941313.526937:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941313.526943:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941313.526946:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17f20. 00000400:00000010:0.0:1788941313.526953:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87908. 00000100:00000200:0.0:1788941313.526960:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591680, portal 25 00000100:00000001:0.0:1788941313.526962:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941313.526964:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768546560 : -131290941005056 : ffff889772196300) 00000100:00000040:0.0:1788941313.526969:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.526976:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941313.526978:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941313.526981:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87990. 00000100:00000200:0.0:1788941313.526983:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1875827959591680, offset 0 00000400:00000200:0.0:1788941313.526989:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941313.526997:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527003:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941313.527008:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941313.527012:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941313.527015:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941313.527023:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527028:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545683:545683:256:4294967295] 192.168.206.112@tcp LPNI seq info [545668:545668:8:4294967295] 00000400:00000200:0.0:1788941313.527033:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527042:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941313.527047:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941313.527051:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84500. 00000800:00000200:0.0:1788941313.527057:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941313.527064:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941313.527069:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941313.527081:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941313.527085:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941313.527087:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 0 is 4307190498 00000100:00000010:0.0:1788941313.527091:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3ea0. 00000100:00000001:0.0:1788941313.527098:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1788941313.527103:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS new deadline estimation for cpu 1 is 4307190498 00000100:00000010:0.0:1788941313.527106:0:2395:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8897480d3960. 00000100:00000001:0.0:1788941313.527110:0:2395:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.206.112@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1788941313.527113:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.527115:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527118:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941313.527122:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527128:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941313.527129:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527131:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941313.527133:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941313.527136:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941313.527138:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591936:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941313.527141:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941313.527149:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941313.527151:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527152:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941313.527155:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748473200. 02000000:00000001:0.0:1788941313.527156:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941313.527160:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941313.527162:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17108. 00000400:00000010:0.0:1788941313.527166:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87330. 00000100:00000200:0.0:1788941313.527169:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591936, portal 10 00000100:00000001:0.0:1788941313.527171:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941313.527172:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768537600 : -131290941014016 : ffff889772194000) 00000100:00000040:0.0:1788941313.527178:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527184:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941313.527186:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941313.527188:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87880. 00000100:00000200:0.0:1788941313.527190:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959591936, offset 0 00000400:00000200:0.0:1788941313.527193:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527197:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527200:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941313.527203:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941313.527205:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941313.527208:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941313.527210:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527213:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545684:545684:255:4294967295] 192.168.206.112@tcp LPNI seq info [545669:545669:7:4294967295] 00000400:00000200:0.0:1788941313.527217:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527223:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941313.527227:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941313.527230:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84200. 00000800:00000200:0.0:1788941313.527234:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941313.527238:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941313.527241:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941313.527248:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.527251:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: using a timeout of 20886 usecs (21 jiffies) 00000400:00000200:0.0:1788941313.527255:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941313.527256:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307186712 00000100:00000001:0.0:1788941313.527259:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527261:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307186712 00000100:00000001:0.0:1788941313.527263:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527265:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.527266:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527269:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527271:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.527278:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941313.527280:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1788941313.527282:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.527283:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941313.527284:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941313.527287:0:2395:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00000001:0.0:1788941313.527288:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.527290:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941313.527291:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.527293:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.527294:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1788941313.527295:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941313.527299:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527305:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941313.527307:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527309:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941313.527310:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941313.527312:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941313.527315:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591808:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941313.527317:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941313.527320:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941313.527325:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527327:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941313.527330:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748605000. 02000000:00000001:0.0:1788941313.527332:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941313.527335:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941313.527338:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17370. 00000400:00000010:0.0:1788941313.527341:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87c38. 00000100:00000200:0.0:1788941313.527344:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959591808, portal 10 00000100:00000001:0.0:1788941313.527346:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941313.527347:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768552832 : -131290940998784 : ffff889772197b80) 00000100:00000040:0.0:1788941313.527353:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527359:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941313.527361:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941313.527363:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd87088. 00000100:00000200:0.0:1788941313.527365:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1875827959591808, offset 0 00000400:00000200:0.0:1788941313.527369:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527372:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527375:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941313.527378:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941313.527380:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941313.527382:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941313.527384:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527387:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545685:545685:254:4294967295] 192.168.206.112@tcp LPNI seq info [545670:545670:6:4294967295] 00000400:00000200:0.0:1788941313.527391:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527397:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941313.527400:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941313.527403:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84900. 00000800:00000200:0.0:1788941313.527407:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941313.527411:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941313.527413:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941313.527421:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.527423:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: using a timeout of 34616 usecs (35 jiffies) 00000400:00000200:0.0:1788941313.527426:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941313.527428:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307186726 00000100:00000001:0.0:1788941313.527430:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527432:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307186726 00000100:00000001:0.0:1788941313.527434:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527436:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.527438:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527439:0:2395:0:(client.c:1813:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1788941313.527443:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527449:0:2395:0:(client.c:1377:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1788941313.527450:0:2395:0:(client.c:1436:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527452:0:2395:0:(sec.c:709:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1788941313.527454:0:2395:0:(sec.c:740:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1788941313.527456:0:2395:0:(sec.c:784:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1788941313.527458:0:2395:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959592064:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941313.527461:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1788941313.527463:0:2395:0:(sec.c:1080:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1788941313.527465:0:2395:0:(sec.c:1117:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.527466:0:2395:0:(sec.c:1855:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1788941313.527469:0:2395:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff889748604600. 02000000:00000001:0.0:1788941313.527471:0:2395:0:(sec.c:1865:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1788941313.527474:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000010:0.0:1788941313.527476:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff889760a17cb8. 00000400:00000010:0.0:1788941313.527480:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd873b8. 00000100:00000200:0.0:1788941313.527482:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1875827959592064, portal 4 00000100:00000001:0.0:1788941313.527498:0:2395:0:(client.c:3275:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1788941313.527499:0:2395:0:(client.c:3277:ptlrpc_request_addref()) Process leaving (rc=18446612782768548352 : -131290941003264 : ffff889772196a00) 00000100:00000040:0.0:1788941313.527506:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527517:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1788941313.527520:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.206.112@tcp 00000400:00000010:0.0:1788941313.527522:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff88976bd877f8. 00000100:00000200:0.0:1788941313.527525:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1875827959592064, offset 0 00000400:00000200:0.0:1788941313.527529:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527532:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527536:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.206.112@tcp local destination 00000400:00000200:0.0:1788941313.527538:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1788941313.527540:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.206.12@tcp 00000400:00000200:0.0:1788941313.527543:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.206.12@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1788941313.527545:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.206.112@tcp 00000400:00000200:0.0:1788941313.527548:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.206.12@tcp NI seq info: [545686:545686:253:4294967295] 192.168.206.112@tcp LPNI seq info [545671:545671:5:4294967295] 00000400:00000200:0.0:1788941313.527552:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.527557:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp:) -> 192.168.206.112@tcp(192.168.206.112@tcp:192.168.206.112@tcp) : PUT try# 0 00000800:00000200:0.0:1788941313.527561:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.206.112@tcp 00000800:00000010:0.0:1788941313.527564:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff88976bd84400. 00000800:00000200:0.0:1788941313.527567:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff889743562900] -> 12345-192.168.206.112@tcp (5) 00000800:00000200:0.0:1788941313.527572:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.206.112@tcp ip 192.168.206.112:988 00000800:00000200:0.0:1788941313.527574:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff88976bd84400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1788941313.527577:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.527580:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: using a timeout of 139171 usecs (140 jiffies) 00000400:00000200:0.0:1788941313.527583:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000100:00000001:0.0:1788941313.527585:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 0 is 4307186831 00000100:00000001:0.0:1788941313.527587:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 0, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527589:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS new deadline estimation for cpu 1 is 4307186831 00000100:00000001:0.0:1788941313.527591:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff889767e18000: PM QoS request active for cpu 1, keep current deadline 4307190498 00000100:00000001:0.0:1788941313.527593:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.527595:0:2395:0:(client.c:1941:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527602:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527604:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527616:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.527618:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1788941313.527657:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941313.527661:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84500. 00000400:00000200:0.0:1788941313.527665:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.527670:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941313.527676:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87990 00000400:00000010:0.0:1788941313.527678:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87990. 00000100:00000001:0.0:1788941313.527691:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941313.527695:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527705:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.527708:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527714:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527716:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941313.527727:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941313.527729:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84200. 00000400:00000200:0.0:1788941313.527732:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.527735:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941313.527737:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87880 00000400:00000010:0.0:1788941313.527738:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87880. 00000100:00000001:0.0:1788941313.527741:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941313.527744:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527751:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.527754:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527759:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527761:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941313.527767:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941313.527774:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84900. 00000400:00000200:0.0:1788941313.527776:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.527780:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941313.527782:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87088 00000400:00000010:0.0:1788941313.527784:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87088. 00000100:00000001:0.0:1788941313.527786:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941313.527789:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527797:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.527800:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527806:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527807:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1788941313.527890:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1788941313.527893:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff88976bd84400. 00000400:00000200:0.0:1788941313.527896:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.527899:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000400:00000200:0.0:1788941313.527902:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd877f8 00000400:00000010:0.0:1788941313.527903:0:2386:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd877f8. 00000100:00000001:0.0:1788941313.527906:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1788941313.527909:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527916:0:2386:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.527918:0:2386:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.527923:0:2386:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.527925:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1788941313.536112:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536129:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941313.536133:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536135:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536143:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536151:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941313.536172:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536177:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536182:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536186:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 25 MB=0x6aa0e3b30f300 00000400:00000200:0.0:1788941313.536190:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536195:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.206.112@tcp of length 224/224 into md 0x538029 [1] + 224 00000800:00000001:0.0:1788941313.536202:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536205:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536208:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.536212:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941313.536219:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536226:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941313.536237:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536251:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941313.536258:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536260:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941313.536262:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536264:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536266:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536270:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941313.536273:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536276:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536278:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536281:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30f400 00000400:00000200:0.0:1788941313.536285:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536288:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x538039 [1] + 224 00000800:00000001:0.0:1788941313.536291:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536293:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536295:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.536302:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941313.536305:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536310:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941313.536318:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536325:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941313.536328:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536330:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941313.536332:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536334:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536336:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536340:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941313.536343:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536346:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536348:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536351:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 4 MB=0x6aa0e3b30f480 00000400:00000200:0.0:1788941313.536354:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536358:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.206.112@tcp of length 224/224 into md 0x538059 [1] + 224 00000800:00000001:0.0:1788941313.536360:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536363:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536370:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.536373:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941313.536376:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536379:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941313.536388:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536394:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941313.536397:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536402:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1788941313.536404:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536406:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536409:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536413:0:2387:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.206.12@tcp(192.168.206.12@tcp) <- 192.168.206.112@tcp : PUT - for me 00000400:00000200:0.0:1788941313.536416:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536419:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536421:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536424:0:2387:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.206.112@tcp of length 224 into portal 10 MB=0x6aa0e3b30f380 00000400:00000200:0.0:1788941313.536427:0:2387:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.206.112@tcp to cpt 0 00000400:00000200:0.0:1788941313.536431:0:2387:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.206.112@tcp of length 224/224 into md 0x538049 [1] + 224 00000800:00000001:0.0:1788941313.536434:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536439:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1788941313.536441:0:2387:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1788941313.536443:0:2387:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.206.12@tcp->192.168.206.112@tcp: PUT: OK 00000100:00000001:0.0:1788941313.536446:0:2387:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536450:0:2387:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1788941313.536458:0:2387:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536465:0:2387:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1788941313.536467:0:2387:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1788941313.536469:0:2387:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.536480:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.536547:0:2395:0:(client.c:1990:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1788941313.536553:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941313.536555:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941313.536558:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17370. 00000400:00000200:0.0:1788941313.536562:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87c38 00000400:00000010:0.0:1788941313.536565:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87c38. 00000100:00000001:0.0:1788941313.536568:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536572:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941313.536591:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536597:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941313.536600:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.536602:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941313.536605:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941313.536607:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941313.536609:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.536612:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941313.536621:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc8ac60 time=111 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941313.536625:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc8ac18 time=5 v=5 (1 1 1 0) 00000100:00000001:0.0:1788941313.536627:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941313.536629:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941313.536632:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941313.536635:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.536638:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941313.536640:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff889767e18000: skip recheck: last_committed 30064771462 00000100:00000001:0.0:1788941313.536642:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941313.536644:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.536648:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536655:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.536658:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941313.536660:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941313.536664:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941313.536676:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772197b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591808:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941313.536689:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941313.536691:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941313.536693:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17cb8. 00000400:00000200:0.0:1788941313.536702:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd873b8 00000400:00000010:0.0:1788941313.536704:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd873b8. 00000100:00000001:0.0:1788941313.536706:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536710:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941313.536717:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536722:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941313.536723:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.536725:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941313.536726:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941313.536728:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941313.536729:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.536731:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941313.536736:0:2395:0:(import.c:2149:obd_at_measure()) add 36 to ffff8897505b5c60 time=110 v=36 (36 5 5 5) 00000100:00001000:0.0:1788941313.536738:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b5c18 time=140 v=5 (1 1 1 1) 00000100:00000001:0.0:1788941313.536741:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941313.536742:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941313.536743:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941313.536745:0:2395:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.536748:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941313.536749:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-OST0000-osc-ffff889767e18000: skip recheck: last_committed 42949693494 00000100:00000001:0.0:1788941313.536751:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941313.536752:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.536755:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536761:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.536763:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941313.536764:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941313.536767:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941313.536774:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959592064:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941313.536798:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941313.536800:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941313.536802:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17f20. 00000400:00000200:0.0:1788941313.536804:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87908 00000400:00000010:0.0:1788941313.536806:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87908. 00000100:00000001:0.0:1788941313.536808:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.536812:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941313.536819:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536824:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941313.536825:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.536827:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941313.536829:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941313.536830:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941313.536831:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.536833:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941313.536836:0:2395:0:(import.c:2149:obd_at_measure()) add 5 to ffff8897505b1460 time=111 v=5 (5 5 5 5) 00000100:00001000:0.0:1788941313.536839:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8897505b1418 time=5 v=5 (1 1 1 0) 00000100:00000001:0.0:1788941313.536841:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941313.536843:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941313.536845:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1788941313.536846:0:2395:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.536848:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.536869:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.536879:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.536881:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941313.536882:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941313.536886:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941313.536893:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772196300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:3b28a452-9a8c-4ca1-ada0-d01d4e7186fd:2395:1875827959591680:192.168.206.112@tcp:400:kworker.0 00000400:00000001:0.0:1788941313.537110:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1788941313.537112:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1788941313.537114:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff889760a17108. 00000400:00000200:0.0:1788941313.537117:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff88976bd87330 00000400:00000010:0.0:1788941313.537118:0:2395:0:(lib-lnet.h:283:lnet_md_free()) slab-freed 'md': 136 at ffff88976bd87330. 00000100:00000001:0.0:1788941313.537121:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1788941313.537125:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1788941313.537134:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537141:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1788941313.537142:0:2395:0:(client.c:3001:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.537144:0:2395:0:(client.c:1593:after_reply()) Process entered 02000000:00000001:0.0:1788941313.537146:0:2395:0:(sec.c:1125:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1788941313.537147:0:2395:0:(pack_generic.c:605:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1788941313.537148:0:2395:0:(pack_generic.c:623:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1788941313.537150:0:2395:0:(sec.c:1181:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1788941313.537154:0:2395:0:(import.c:2149:obd_at_measure()) add 60 to ffff88976bc88460 time=111 v=60 (60 60 60 61) 00000100:00001000:0.0:1788941313.537157:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff88976bc88418 time=125 v=5 (1 1 0 0) 00000100:00000001:0.0:1788941313.537160:0:2395:0:(client.c:1495:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1788941313.537161:0:2395:0:(client.c:1533:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1788941313.537163:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1788941313.537166:0:2395:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1788941313.537173:0:2395:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.537176:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1788941313.537177:0:2395:0:(client.c:3121:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff889767e18000: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1788941313.537179:0:2395:0:(client.c:3122:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1788941313.537180:0:2395:0:(client.c:1797:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1788941313.537184:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537190:0:2395:0:(client.c:2354:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1788941313.537197:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1788941313.537199:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1788941313.537204:0:2395:0:(lustre_net.h:2405:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1788941313.537212:0:2395:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff889772194000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:5577a8ea-8937-4a4d-9ce6-14bd397bfba4:2395:1875827959591936:192.168.206.112@tcp:400:kworker.0 00000100:00000001:0.0:1788941313.537216:0:2395:0:(client.c:2465:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.537219:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.537223:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772197b80 x1875827959591808/t0(0) o400->lustre-MDT0000-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537231:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941313.537233:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941313.537237:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748605000. 02000000:00000001:0.0:1788941313.537241:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941313.537244:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941313.537246:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc8a800 refcount=5 obd=lustre-MDT0000-mdc-ffff889767e18000 00000020:00000001:0.0:1788941313.537249:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941313.537252:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5500. 02000000:00000001:0.0:1788941313.537256:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941313.537257:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941313.537259:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772197b80. 00000100:00000001:0.0:1788941313.537263:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941313.537264:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.537266:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.537269:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196a00 x1875827959592064/t0(0) o400->lustre-OST0000-osc-ffff889767e18000@192.168.206.112@tcp:28/4 lens 224/224 e 0 to 0 dl 1788941368 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537276:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941313.537277:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941313.537278:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748604600. 02000000:00000001:0.0:1788941313.537281:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941313.537282:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941313.537283:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b5800 refcount=3 obd=lustre-OST0000-osc-ffff889767e18000 00000020:00000001:0.0:1788941313.537285:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941313.537286:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5700. 02000000:00000001:0.0:1788941313.537293:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941313.537295:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941313.537296:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196a00. 00000100:00000001:0.0:1788941313.537299:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941313.537301:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.537302:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.537306:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772196300 x1875827959591680/t0(0) o400->MGC192.168.206.112@tcp@192.168.206.112@tcp:26/25 lens 224/224 e 0 to 0 dl 1788941329 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537313:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941313.537314:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941313.537316:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748472200. 02000000:00000001:0.0:1788941313.537319:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941313.537320:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941313.537322:0:2395:0:(genops.c:1223:class_import_put()) import ffff8897505b1000 refcount=3 obd=MGC192.168.206.112@tcp 00000020:00000001:0.0:1788941313.537324:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941313.537325:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5b00. 02000000:00000001:0.0:1788941313.537328:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941313.537329:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941313.537330:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772196300. 00000100:00000001:0.0:1788941313.537332:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941313.537334:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.537336:0:2395:0:(client.c:2867:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1788941313.537340:0:2395:0:(client.c:2875:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff889772194000 x1875827959591936/t0(0) o400->lustre-MDT0001-mdc-ffff889767e18000@192.168.206.112@tcp:12/10 lens 224/224 e 0 to 0 dl 1788941398 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1788941313.537348:0:2395:0:(client.c:2793:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1788941313.537350:0:2395:0:(sec.c:1880:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1788941313.537352:0:2395:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff889748473200. 02000000:00000001:0.0:1788941313.537354:0:2395:0:(sec.c:1896:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1788941313.537355:0:2395:0:(genops.c:1217:class_import_put()) Process entered 00000020:00000040:0.0:1788941313.537357:0:2395:0:(genops.c:1223:class_import_put()) import ffff88976bc88000 refcount=3 obd=lustre-MDT0001-mdc-ffff889767e18000 00000020:00000001:0.0:1788941313.538017:0:2395:0:(genops.c:1230:class_import_put()) Process leaving 02000000:00000010:0.0:1788941313.538021:0:2395:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff88976d0d5e00. 02000000:00000001:0.0:1788941313.538026:0:2395:0:(sec.c:473:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1788941313.538028:0:2395:0:(sec.c:490:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1788941313.538029:0:2395:0:(client.c:612:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff889772194000. 00000100:00000001:0.0:1788941313.538032:0:2395:0:(client.c:2849:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1788941313.538040:0:2395:0:(client.c:2908:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.538043:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1788941313.538050:0:2395:0:(client.c:2639:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1788941313.538052:0:2395:0:(client.c:2667:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1788941313.538054:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1788941313.538057:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941313.785842:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941313.785882:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941313.785891:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941313.785893:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941313.785980:0:245732:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941313.785983:0:245732:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941313.785987:0:245732:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:2.0:1788941314.803543:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941314.803554:0:250116:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788941314.803557:0:250116:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788941314.803572:0:250116:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803577:0:250116:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788941314.803579:0:250116:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803582:0:250116:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803585:0:250116:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803590:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941314.803592:0:250116:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:2.0:1788941314.803593:0:250116:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:2.0:1788941314.803599:0:250116:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:2.0:1788941314.803601:0:250116:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:2.0:1788941314.803602:0:250116:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803604:0:250116:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:2.0:1788941314.803605:0:250116:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803607:0:250116:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941314.803608:0:250116:0:(ldlm_pool.c:496:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1788941314.804263:0:242292:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1788941314.804267:0:242292:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1788941314.804272:0:242292:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1788941315.582921:0:278844:0:(llite_lib.c:4247:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941315.826926:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941315.826931:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941315.826936:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941315.826937:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1788941315.827049:0:252129:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1788941315.827053:0:252129:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1788941315.827055:0:252129:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000001:0.0:1788941315.892735:0:278880:0:(debug.c:652:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:0.0:1788941315.892741:0:278880:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_812a: @@@@@@ FAIL: client: import is not in CONNECTING state after 50 00000001:00000001:0.0:1788941315.898661:0:278880:0:(debug.c:655:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:2.0:1788941316.851061:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941316.851067:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1788941316.851072:0:250116:0:(ldlm_pool.c:457:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1788941316.851073:0:250116:0:(ldlm_pool.c:461:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1788941316.851098:0:236344:0:(genops.c:1895:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1788941316.851101:0:236344:0:(genops.c:1909:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1788941316.851103:0:236344:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:3.0:1788941317.274004:0:278980:0:(llite_lib.c:4247:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1788941317.362990:0:276806:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af110 (cpu 1) has reached its deadline 4307190498, at 4307190528 00000100:00000010:1.0:1788941317.363001:0:276806:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3960. 00000100:00000040:0.0:1788941317.364633:0:265097:0:(connection.c:75:cpu_latency_work()) work item of ffff8897466af000 (cpu 0) has reached its deadline 4307190498, at 4307190529 00000100:00000010:0.0:1788941317.364642:0:265097:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8897480d3ea0. Debug log: 2633 lines, 2633 kept, 0 dropped, 0 bad.