00000100:00000040:0.0F:1787148436.000891:0:231971:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342000 (cpu 0) has reached its deadline 4306944761, at 4306944770 00000100:00000010:0.0:1787148436.000932:0:231971:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c650cc120. 00000100:00000040:1.0F:1787148436.001028:0:256195:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342110 (cpu 1) has reached its deadline 4306944761, at 4306944770 00000100:00000010:1.0:1787148436.001041:0:256195:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c650cc300. 00010000:00000001:1.0:1787148436.446248:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148436.446256:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148436.446262:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148436.446263:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787148436.446288:0:235349:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787148436.446291:0:235349:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787148436.446295:0:235349:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:3.0F:1787148437.279397:0:233104:0:(pinger.c:229:ptlrpc_pinger_process_import()) adab7261-4d97-43ea-82e8-09637750fa21->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787148437.279414:0:233104:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787148437.279421:0:233104:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c72650000. 00000020:00000040:3.0:1787148437.279426:0:233104:0:(genops.c:1231:class_import_get()) import ffff8c5c5776b000 refcount=4 obd=MGC192.168.202.114@tcp 00000100:00000001:3.0:1787148437.279431:0:233104:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787148437.279434:0:233104:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787148437.279439:0:233104:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787148437.279447:0:233104:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c50a4e400. 00000100:00000001:3.0:1787148437.279454:0:233104:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279466:0:233104:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging adab7261-4d97-43ea-82e8-09637750fa21->MGS req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787148437.279477:0:233104:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787148437.279482:0:233104:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279488:0:233104:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007573b71b] to pc [ptlrpcd_01_00+0] req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279519:0:233104:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279522:0:233104:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787148437.279525:0:233104:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787148437.279528:0:233104:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c72651f80. 00000020:00000040:3.0:1787148437.279531:0:233104:0:(genops.c:1231:class_import_get()) import ffff8c5c5776a000 refcount=6 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000100:00000001:3.0:1787148437.279533:0:233104:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787148437.279535:0:233104:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787148437.279536:0:233104:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787148437.279540:0:233104:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c50a4e000. 00000100:00000001:3.0:1787148437.279543:0:233104:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279547:0:233104:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787148437.279554:0:233104:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787148437.279556:0:233104:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279560:0:233104:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007ab4cd0c] to pc [ptlrpcd_01_01+1] req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279567:0:233104:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279576:0:233104:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787148437.279579:0:233104:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787148437.279582:0:233104:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c72653800. 00000020:00000040:3.0:1787148437.279585:0:233104:0:(genops.c:1231:class_import_get()) import ffff8c5c5776c800 refcount=4 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000100:00000001:3.0:1787148437.279587:0:233104:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787148437.279588:0:233104:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787148437.279590:0:233104:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787148437.279594:0:233104:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c50a4e700. 00000100:00000001:3.0:1787148437.279597:0:233104:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279601:0:233104:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787148437.279608:0:233104:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787148437.279610:0:233104:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279614:0:233104:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000000a403de5] to pc [ptlrpcd_01_00+0] req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279620:0:233104:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279622:0:233104:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:3.0:1787148437.279625:0:233104:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:3.0:1787148437.279628:0:233104:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c72650700. 00000020:00000040:3.0:1787148437.279630:0:233104:0:(genops.c:1231:class_import_get()) import ffff8c5c5e0b1800 refcount=4 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000100:00000001:3.0:1787148437.279632:0:233104:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:3.0:1787148437.279633:0:233104:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:3.0:1787148437.279634:0:233104:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1787148437.279638:0:233104:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c50a4e100. 00000100:00000001:3.0:1787148437.279641:0:233104:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279644:0:233104:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:3.0:1787148437.279650:0:233104:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:3.0:1787148437.279652:0:233104:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279655:0:233104:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005602ac3f] to pc [ptlrpcd_01_01+1] req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279672:0:233104:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279674:0:233104:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:3.0:1787148437.279677:0:233104:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:3.0:1787148437.279681:0:233104:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12283) 00000100:00000001:3.0:1787148437.279698:0:2396:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.279701:0:2396:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787148437.279703:0:2396:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787148437.279709:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279717:0:2396:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787148437.279718:0:2396:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.279722:0:2396:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787148437.279725:0:2396:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787148437.279727:0:2396:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787148437.279756:0:2396:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c72651f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2396:1873948276730752:192.168.202.114@tcp:400:kworker.0 00000100:00000001:3.0:1787148437.279761:0:2396:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787148437.279766:0:2396:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787148437.279768:0:2396:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.279771:0:2396:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787148437.279775:0:2396:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c58ae6000. 02000000:00000001:3.0:1787148437.279777:0:2396:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787148437.279785:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:3.0:1787148437.279788:0:2396:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c4a93f6e0. 00000400:00000010:3.0:1787148437.279796:0:2396:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c72962088. 00000100:00000200:3.0:1787148437.279803:0:2396:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276730752, portal 10 00000100:00000001:3.0:1787148437.279806:0:2396:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787148437.279807:0:2396:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927416950656 : -127146292600960 : ffff8c5c72651f80) 00000100:00000040:3.0:1787148437.279814:0:2396:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279820:0:2396:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787148437.279829:0:2396:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:3.0:1787148437.279831:0:2396:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c72962770. 00000100:00000200:3.0:1787148437.279835:0:2396:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276730752, offset 0 00000400:00000200:3.0:1787148437.279841:0:2396:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:3.0:1787148437.279850:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:3.0:1787148437.279857:0:2396:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:3.0:1787148437.279862:0:2396:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787148437.279866:0:2396:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:3.0:1787148437.279869:0:2396:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787148437.279872:0:2396:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:3.0:1787148437.279875:0:2396:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556886:556886:256:4294967295] 192.168.202.114@tcp LPNI seq info [556842:556842:8:4294967295] 00000400:00000200:3.0:1787148437.279879:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:3.0:1787148437.279888:0:2396:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787148437.279894:0:2396:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:3.0:1787148437.279897:0:2396:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c50a4e200. 00000800:00000200:3.0:1787148437.279904:0:2396:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:3.0:1787148437.279911:0:2396:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:3.0:1787148437.279916:0:2396:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c50a4e200 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787148437.279928:0:2396:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.279933:0:2396:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: using a timeout of 24715 usecs (25 jiffies) 00000400:00000200:3.0:1787148437.279936:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:3.0:1787148437.279939:0:2396:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306946074 00000100:00000010:3.0:1787148437.279942:0:2396:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c650cc6c0. 00000100:00000001:3.0:1787148437.279949:0:2396:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request now active for cpu 0 00000100:00000001:3.0:1787148437.279955:0:2396:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306946074 00000100:00000010:3.0:1787148437.279958:0:2396:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c650cc180. 00000100:00000001:3.0:1787148437.279961:0:2396:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request now active for cpu 1 00000100:00000001:3.0:1787148437.279964:0:2396:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787148437.279967:0:2396:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.279970:0:2396:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:3.0:1787148437.279979:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.279985:0:2396:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1787148437.279986:0:2396:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.279988:0:2396:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1787148437.279990:0:2396:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1787148437.279992:0:2396:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:3.0:1787148437.279995:0:2396:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c72650700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2396:1873948276731008:192.168.202.114@tcp:400:kworker.0 00000100:00000001:3.0:1787148437.279998:0:2396:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:3.0:1787148437.280000:0:2396:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:3.0:1787148437.280014:0:2396:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.280016:0:2396:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:3.0:1787148437.280019:0:2396:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c58ae7800. 02000000:00000001:3.0:1787148437.280021:0:2396:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:3.0:1787148437.280025:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:3.0:1787148437.280028:0:2396:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c4a93f0b0. 00000400:00000010:3.0:1787148437.280032:0:2396:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c72962ee0. 00000100:00000200:3.0:1787148437.280035:0:2396:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731008, portal 4 00000100:00000001:3.0:1787148437.280037:0:2396:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:3.0:1787148437.280038:0:2396:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927416944384 : -127146292607232 : ffff8c5c72650700) 00000100:00000040:3.0:1787148437.280044:0:2396:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.280049:0:2396:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1787148437.280051:0:2396:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:3.0:1787148437.280053:0:2396:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c72962880. 00000100:00000200:3.0:1787148437.280055:0:2396:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1873948276731008, offset 0 00000400:00000200:3.0:1787148437.280059:0:2396:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:3.0:1787148437.280062:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:3.0:1787148437.280066:0:2396:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:3.0:1787148437.280069:0:2396:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1787148437.280071:0:2396:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:3.0:1787148437.280073:0:2396:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1787148437.280080:0:2396:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:3.0:1787148437.280084:0:2396:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556887:556887:255:4294967295] 192.168.202.114@tcp LPNI seq info [556843:556843:7:4294967295] 00000400:00000200:3.0:1787148437.280087:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:3.0:1787148437.280093:0:2396:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:3.0:1787148437.280098:0:2396:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:3.0:1787148437.280101:0:2396:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c50a4ef00. 00000800:00000200:3.0:1787148437.280124:0:2396:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:3.0:1787148437.280128:0:2396:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:3.0:1787148437.280131:0:2396:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c50a4ef00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1787148437.280134:0:2396:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.280137:0:2396:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: using a timeout of 120892 usecs (121 jiffies) 00000400:00000200:3.0:1787148437.280139:0:2396:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:3.0:1787148437.280141:0:2396:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306946170 00000100:00000001:3.0:1787148437.280146:0:2396:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 0, simply extend its deadline from 4306946074 00000100:00000001:3.0:1787148437.280150:0:2396:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306946170 00000100:00000001:3.0:1787148437.280152:0:2396:0:(niobuf.c:685:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 1, simply extend its deadline from 4306946074 00000100:00000001:3.0:1787148437.280155:0:2396:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0F:1787148437.280304:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787148437.280313:0:2395:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787148437.280314:0:2395:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787148437.280321:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787148437.280328:0:2395:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787148437.280330:0:2395:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787148437.280332:0:2395:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787148437.280334:0:2395:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787148437.280336:0:2395:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787148437.280339:0:2395:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c72650000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:adab7261-4d97-43ea-82e8-09637750fa21:2395:1873948276730624:192.168.202.114@tcp:400:kworker.0 00000100:00000001:2.0:1787148437.280342:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787148437.280345:0:2395:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787148437.280346:0:2395:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787148437.280348:0:2395:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787148437.280352:0:2395:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4ac87800. 02000000:00000001:2.0:1787148437.280353:0:2395:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787148437.280358:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:2.0:1787148437.280361:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c68ddbf78. 00000400:00000010:2.0:1787148437.280364:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c42ee3220. 00000100:00000200:2.0:1787148437.280367:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276730624, portal 25 00000100:00000001:2.0:1787148437.280369:0:2395:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787148437.280370:0:2395:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927416942592 : -127146292609024 : ffff8c5c72650000) 00000100:00000040:2.0:1787148437.280375:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787148437.280381:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787148437.280383:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:2.0:1787148437.280385:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c42ee3088. 00000100:00000200:2.0:1787148437.280388:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1873948276730624, offset 0 00000400:00000200:2.0:1787148437.280392:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:2.0:1787148437.280396:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:2.0:1787148437.280400:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:2.0:1787148437.280402:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787148437.280405:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:2.0:1787148437.280414:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787148437.280416:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:2.0:1787148437.280419:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556888:556888:254:4294967295] 192.168.202.114@tcp LPNI seq info [556844:556844:6:4294967295] 00000400:00000200:2.0:1787148437.280423:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:2.0:1787148437.280429:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787148437.280434:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:2.0:1787148437.280438:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c46fcb400. 00000800:00000200:2.0:1787148437.280442:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:2.0:1787148437.280446:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:2.0:1787148437.280449:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c46fcb400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787148437.280452:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787148437.280455:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:2.0:1787148437.280457:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 0 is 4306949883 00000100:00000001:2.0:1787148437.280461:0:2395:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306946170 00000100:00000001:2.0:1787148437.280464:0:2395:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 1 is 4306949883 00000100:00000001:2.0:1787148437.280466:0:2395:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306946170 00000100:00000001:2.0:1787148437.280469:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787148437.280471:0:2395:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787148437.280472:0:2395:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:2.0:1787148437.280476:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787148437.280483:0:2395:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:2.0:1787148437.280484:0:2395:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787148437.280486:0:2395:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:2.0:1787148437.280488:0:2395:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:2.0:1787148437.280490:0:2395:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:2.0:1787148437.280493:0:2395:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c72653800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2395:1873948276730880:192.168.202.114@tcp:400:kworker.0 00000100:00000001:2.0:1787148437.280496:0:2395:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:2.0:1787148437.280499:0:2395:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:2.0:1787148437.280507:0:2395:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1787148437.280508:0:2395:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:2.0:1787148437.280511:0:2395:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4ac87e00. 02000000:00000001:2.0:1787148437.280513:0:2395:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:2.0:1787148437.280516:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:2.0:1787148437.280518:0:2395:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c68ddb840. 00000400:00000010:2.0:1787148437.280522:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c42ee3e58. 00000100:00000200:2.0:1787148437.280524:0:2395:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276730880, portal 10 00000100:00000001:2.0:1787148437.280526:0:2395:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:2.0:1787148437.280528:0:2395:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927416956928 : -127146292594688 : ffff8c5c72653800) 00000100:00000040:2.0:1787148437.280533:0:2395:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1787148437.280540:0:2395:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:2.0:1787148437.280542:0:2395:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:2.0:1787148437.280544:0:2395:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c42ee3550. 00000100:00000200:2.0:1787148437.280546:0:2395:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276730880, offset 0 00000400:00000200:2.0:1787148437.280551:0:2395:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:2.0:1787148437.280555:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:2.0:1787148437.280558:0:2395:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:2.0:1787148437.280560:0:2395:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:2.0:1787148437.280563:0:2395:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:2.0:1787148437.280565:0:2395:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:2.0:1787148437.280567:0:2395:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:2.0:1787148437.280569:0:2395:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556889:556889:253:4294967295] 192.168.202.114@tcp LPNI seq info [556845:556845:5:4294967295] 00000400:00000200:2.0:1787148437.280573:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:2.0:1787148437.280578:0:2395:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:2.0:1787148437.280583:0:2395:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:2.0:1787148437.280586:0:2395:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c46fcbb00. 00000800:00000200:2.0:1787148437.280590:0:2395:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:2.0:1787148437.280594:0:2395:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:2.0:1787148437.280597:0:2395:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c46fcbb00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:2.0:1787148437.280605:0:2395:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1787148437.280608:0:2395:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: using a timeout of 16988 usecs (17 jiffies) 00000400:00000200:2.0:1787148437.280612:0:2395:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:2.0:1787148437.280614:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306946067 00000100:00000001:2.0:1787148437.280616:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306949883 00000100:00000001:2.0:1787148437.280618:0:2395:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306946067 00000100:00000001:2.0:1787148437.280620:0:2395:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306949883 00000100:00000001:2.0:1787148437.280622:0:2395:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1787148437.280624:0:2395:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787148437.280626:0:2395:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787148437.280628:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1787148437.280635:0:2395:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1787148437.280637:0:2395:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:2.0:1787148437.280639:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1787148437.280640:0:2395:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:2.0:1787148437.280641:0:2395:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1787148437.280644:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.280696:0:2396:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.280699:0:2396:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.280701:0:2396:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.280705:0:2396:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787148437.280706:0:2396:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:3.0:1787148437.280708:0:2396:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.280709:0:2396:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787148437.280711:0:2396:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.280712:0:2396:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787148437.281250:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148437.281255:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c50a4e200. 00000400:00000200:0.0:1787148437.281264:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148437.281270:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148437.281276:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c72962770 00000400:00000010:0.0:1787148437.281278:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c72962770. 00000100:00000001:0.0:1787148437.281283:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148437.281289:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281301:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148437.281305:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281311:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148437.281313:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1787148437.281326:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148437.281329:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c50a4ef00. 00000400:00000200:0.0:1787148437.281331:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148437.281334:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148437.281337:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c72962880 00000400:00000010:0.0:1787148437.281338:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c72962880. 00000100:00000001:0.0:1787148437.281340:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148437.281344:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281351:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148437.281354:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281360:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148437.281361:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1787148437.281369:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148437.281371:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c46fcb400. 00000400:00000200:0.0:1787148437.281374:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148437.281384:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148437.281387:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c42ee3088 00000400:00000010:0.0:1787148437.281389:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c42ee3088. 00000100:00000001:0.0:1787148437.281392:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148437.281396:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281402:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148437.281405:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281411:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148437.281412:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1787148437.281476:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148437.281479:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c46fcbb00. 00000400:00000200:0.0:1787148437.281482:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148437.281485:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148437.281487:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c42ee3550 00000400:00000010:0.0:1787148437.281489:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c42ee3550. 00000100:00000001:0.0:1787148437.281491:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148437.281494:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281500:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148437.281502:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148437.281507:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148437.281508:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1787148437.287057:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.287078:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148437.287083:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.287086:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.287097:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287131:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148437.287144:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287149:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287155:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287172:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 25 MB=0x6a858955cab00 00000400:00000200:1.0:1787148437.287177:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287184:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561dc1 [1] + 224 00000800:00000001:1.0:1787148437.287193:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.287198:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.287202:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148437.287208:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148437.287216:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148437.287227:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148437.287240:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148437.287270:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148437.287281:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.287285:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148437.287288:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.287307:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.287313:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287317:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148437.287324:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.287327:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:3.0:1787148437.287329:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:1.0:1787148437.287333:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:3.0:1787148437.287336:0:2395:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000200:1.0:1787148437.287340:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 4 MB=0x6a858955cac80 00000400:00000001:3.0:1787148437.287345:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:1.0:1787148437.287345:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000001:3.0:1787148437.287349:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787148437.287353:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c68ddbf78. 00000400:00000200:3.0:1787148437.287366:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c42ee3220 00000400:00000010:3.0:1787148437.287376:0:2395:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c42ee3220. 00000100:00000001:3.0:1787148437.287380:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787148437.287385:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787148437.287397:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.287403:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787148437.287405:0:2395:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.287408:0:2395:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1787148437.287411:0:2395:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787148437.287413:0:2395:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787148437.287415:0:2395:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.287418:0:2395:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787148437.287425:0:2395:0:(import.c:2149:obd_at_measure()) add 5 to ffff8c5c5776b460 time=69 v=5 (5 5 5 5) 00000100:00001000:3.0:1787148437.287429:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776b418 time=121 v=5 (1 1 0 0) 00000100:00000001:3.0:1787148437.287432:0:2395:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787148437.287433:0:2395:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787148437.287436:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787148437.287438:0:2395:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.287441:0:2395:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.287445:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.287451:0:2395:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787148437.287454:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787148437.287456:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787148437.287460:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787148437.287467:0:2395:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c72650000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:adab7261-4d97-43ea-82e8-09637750fa21:2395:1873948276730624:192.168.202.114@tcp:400:kworker.0 00000100:00000001:3.0:1787148437.287472:0:2395:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.287475:0:2395:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787148437.287478:0:2395:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c72650000 x1873948276730624/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148453 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.287494:0:2395:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787148437.287497:0:2395:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787148437.287501:0:2395:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4ac87800. 02000000:00000001:3.0:1787148437.287515:0:2395:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787148437.287517:0:2395:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1787148437.288172:0:2395:0:(genops.c:1244:class_import_put()) import ffff8c5c5776b000 refcount=3 obd=MGC192.168.202.114@tcp 00000020:00000001:3.0:1787148437.288177:0:2395:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1787148437.288180:0:2395:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c50a4e400. 02000000:00000001:3.0:1787148437.288185:0:2395:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787148437.288187:0:2395:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787148437.288190:0:2395:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c72650000. 00000100:00000001:3.0:1787148437.288193:0:2395:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787148437.288195:0:2395:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.288199:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.288202:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.288203:0:2395:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1787148437.288205:0:2395:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.288207:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148437.294078:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561db9 [1] + 224 00000800:00000001:1.0:1787148437.294099:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294131:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.294136:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148437.294142:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148437.294163:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148437.294174:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148437.294186:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148437.294210:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148437.294223:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294227:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148437.294230:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294232:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.294239:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294243:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148437.294249:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294253:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294257:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294261:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cab80 00000400:00000200:1.0:1787148437.294265:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294270:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561db1 [1] + 224 00000800:00000001:1.0:1787148437.294274:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294287:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.294289:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148437.294293:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148437.294297:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148437.294303:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148437.294313:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148437.294321:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148437.294325:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294329:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148437.294331:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294333:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.294336:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294340:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148437.294344:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294347:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294349:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294352:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cac00 00000400:00000200:1.0:1787148437.294355:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148437.294359:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561dc9 [1] + 224 00000800:00000001:1.0:1787148437.294362:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294372:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148437.294375:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148437.294378:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148437.294381:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148437.294385:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148437.294393:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148437.294405:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148437.294416:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148437.294419:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.294808:0:2396:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.294813:0:2396:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787148437.294821:0:2396:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787148437.294824:0:2396:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787148437.294828:0:2396:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c4a93f6e0. 00000400:00000200:3.0:1787148437.294834:0:2396:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c72962088 00000400:00000010:3.0:1787148437.294837:0:2396:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c72962088. 00000100:00000001:3.0:1787148437.294842:0:2396:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787148437.294848:0:2396:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787148437.294858:0:2396:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.294866:0:2396:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787148437.294868:0:2396:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.294871:0:2396:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1787148437.294875:0:2396:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787148437.294877:0:2396:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787148437.294887:0:2396:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.294890:0:2396:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787148437.294903:0:2396:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776a460 time=69 v=36 (36 36 36 36) 00000100:00001000:3.0:1787148437.294908:0:2396:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776a418 time=121 v=5 (1 1 0 0) 00000100:00000001:3.0:1787148437.294911:0:2396:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787148437.294913:0:2396:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787148437.294916:0:2396:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787148437.294921:0:2396:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.294926:0:2396:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787148437.294928:0:2396:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8c5c68fb1800: skip recheck: last_committed 30064771460 00000100:00000001:3.0:1787148437.294931:0:2396:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787148437.294934:0:2396:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.294939:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.294947:0:2396:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787148437.294950:0:2396:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787148437.294953:0:2396:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787148437.294957:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787148437.294978:0:2396:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c72651f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2396:1873948276730752:192.168.202.114@tcp:400:kworker.0 00000400:00000001:3.0:1787148437.294987:0:2396:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787148437.294989:0:2396:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787148437.294991:0:2396:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c4a93f0b0. 00000400:00000200:3.0:1787148437.294994:0:2396:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c72962ee0 00000400:00000010:3.0:1787148437.294996:0:2396:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c72962ee0. 00000100:00000001:3.0:1787148437.294999:0:2396:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787148437.295003:0:2396:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787148437.295011:0:2396:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295017:0:2396:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787148437.295036:0:2396:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295045:0:2396:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1787148437.295047:0:2396:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787148437.295049:0:2396:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787148437.295050:0:2396:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.295052:0:2396:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787148437.295057:0:2396:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5e0b1c60 time=69 v=36 (36 5 5 5) 00000100:00001000:3.0:1787148437.295060:0:2396:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5e0b1c18 time=45 v=5 (1 1 1 1) 00000100:00000001:3.0:1787148437.295062:0:2396:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787148437.295064:0:2396:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787148437.295065:0:2396:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:3.0:1787148437.295067:0:2396:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.295069:0:2396:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787148437.295071:0:2396:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8c5c68fb1800: skip recheck: last_committed 42949695890 00000100:00000001:3.0:1787148437.295072:0:2396:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787148437.295074:0:2396:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.295078:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295093:0:2396:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787148437.295094:0:2396:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787148437.295096:0:2396:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787148437.295099:0:2396:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787148437.295127:0:2396:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c72650700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2396:1873948276731008:192.168.202.114@tcp:400:kworker.0 00000100:00000001:3.0:1787148437.295133:0:2396:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295136:0:2396:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787148437.295140:0:2396:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c72651f80 x1873948276730752/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295148:0:2396:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787148437.295151:0:2396:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787148437.295154:0:2396:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c58ae6000. 02000000:00000001:3.0:1787148437.295159:0:2396:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787148437.295161:0:2396:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1787148437.295168:0:2396:0:(genops.c:1244:class_import_put()) import ffff8c5c5776a000 refcount=5 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000020:00000001:3.0:1787148437.295170:0:2396:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1787148437.295173:0:2396:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c50a4e000. 02000000:00000001:3.0:1787148437.295176:0:2396:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787148437.295178:0:2396:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787148437.295180:0:2396:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c72651f80. 00000100:00000001:3.0:1787148437.295184:0:2396:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787148437.295185:0:2396:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295187:0:2396:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787148437.295191:0:2396:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c72650700 x1873948276731008/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295197:0:2396:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787148437.295198:0:2396:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787148437.295200:0:2396:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c58ae7800. 02000000:00000001:3.0:1787148437.295202:0:2396:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787148437.295203:0:2396:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1787148437.295204:0:2396:0:(genops.c:1244:class_import_put()) import ffff8c5c5e0b1800 refcount=3 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000020:00000001:3.0:1787148437.295206:0:2396:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1787148437.295207:0:2396:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c50a4e100. 02000000:00000001:3.0:1787148437.295209:0:2396:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787148437.295210:0:2396:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787148437.295221:0:2396:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c72650700. 00000100:00000001:3.0:1787148437.295224:0:2396:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787148437.295225:0:2396:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295228:0:2396:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295237:0:2396:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787148437.295239:0:2396:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.295242:0:2396:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.295245:0:2396:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.295256:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.295259:0:2395:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1787148437.295264:0:2395:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:3.0:1787148437.295266:0:2395:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1787148437.295268:0:2395:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c68ddb840. 00000400:00000200:3.0:1787148437.295272:0:2395:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c42ee3e58 00000400:00000010:3.0:1787148437.295810:0:2395:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c42ee3e58. 00000100:00000001:3.0:1787148437.295819:0:2395:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:3.0:1787148437.295831:0:2395:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:3.0:1787148437.295841:0:2395:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295848:0:2395:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1787148437.295850:0:2395:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295852:0:2395:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:3.0:1787148437.295854:0:2395:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1787148437.295856:0:2395:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1787148437.295858:0:2395:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1787148437.295860:0:2395:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:3.0:1787148437.295866:0:2395:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776cc60 time=69 v=36 (36 36 36 36) 00000100:00001000:3.0:1787148437.295870:0:2395:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776cc18 time=96 v=5 (1 1 0 0) 00000100:00000001:3.0:1787148437.295872:0:2395:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1787148437.295874:0:2395:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:3.0:1787148437.295877:0:2395:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:3.0:1787148437.295881:0:2395:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:3.0:1787148437.295888:0:2395:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.295891:0:2395:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:3.0:1787148437.295893:0:2395:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8c5c68fb1800: skip recheck: last_committed 21474836540 00000100:00000001:3.0:1787148437.295895:0:2395:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:3.0:1787148437.295897:0:2395:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1787148437.295901:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295908:0:2395:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1787148437.295911:0:2395:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1787148437.295912:0:2395:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1787148437.295915:0:2395:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1787148437.295922:0:2395:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c72653800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2395:1873948276730880:192.168.202.114@tcp:400:kworker.0 00000100:00000001:3.0:1787148437.295935:0:2395:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295937:0:2395:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1787148437.295941:0:2395:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c72653800 x1873948276730880/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148492 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1787148437.295947:0:2395:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1787148437.295949:0:2395:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:3.0:1787148437.295952:0:2395:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4ac87e00. 02000000:00000001:3.0:1787148437.295956:0:2395:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:3.0:1787148437.295958:0:2395:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:3.0:1787148437.295959:0:2395:0:(genops.c:1244:class_import_put()) import ffff8c5c5776c800 refcount=3 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000020:00000001:3.0:1787148437.295975:0:2395:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:3.0:1787148437.295977:0:2395:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c50a4e700. 02000000:00000001:3.0:1787148437.295981:0:2395:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:3.0:1787148437.295983:0:2395:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1787148437.295985:0:2395:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c72653800. 00000100:00000001:3.0:1787148437.295988:0:2395:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1787148437.295990:0:2395:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295992:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1787148437.295997:0:2395:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1787148437.295999:0:2395:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1787148437.296001:0:2395:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1787148437.296003:0:2395:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148437.470962:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148437.470971:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148437.470977:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148437.470979:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148437.471827:0:244719:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148437.471832:0:244719:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148437.471835:0:244719:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148438.494234:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148438.494241:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148438.494248:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148438.494249:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148438.497020:0:244722:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148438.497028:0:244722:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148438.497033:0:244722:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148439.520467:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148439.520476:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148439.520482:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148439.520483:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787148439.521292:0:256812:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787148439.521297:0:256812:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787148439.521302:0:256812:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148440.542913:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148440.542922:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148440.542929:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148440.542931:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148440.543011:0:235348:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148440.543015:0:235348:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148440.543019:0:235348:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787148441.118273:0:231971:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342000 (cpu 0) has reached its deadline 4306949883, at 4306949888 00000100:00000010:0.0:1787148441.118293:0:231971:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c650cc6c0. 00000100:00000040:1.0:1787148441.119170:0:256195:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342110 (cpu 1) has reached its deadline 4306949883, at 4306949888 00000100:00000010:1.0:1787148441.119180:0:256195:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c650cc180. 00010000:00000001:1.0:1787148441.566255:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148441.566264:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148441.566272:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148441.566276:0:256195:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1787148441.566278:0:256195:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1787148441.566285:0:256195:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1787148441.566287:0:256195:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1787148441.566289:0:256195:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148441.566292:0:256195:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1787148441.566294:0:256195:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148441.566297:0:256195:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148441.566299:0:256195:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787148441.566329:0:235349:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787148441.566332:0:235349:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787148441.566335:0:235349:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1787148442.398344:0:217321:0:(pinger.c:229:ptlrpc_pinger_process_import()) adab7261-4d97-43ea-82e8-09637750fa21->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787148442.398356:0:217321:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787148442.398381:0:217321:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6f339c00. 00000020:00000040:1.0:1787148442.398388:0:217321:0:(genops.c:1231:class_import_get()) import ffff8c5c5776b000 refcount=4 obd=MGC192.168.202.114@tcp 00000100:00000001:1.0:1787148442.398392:0:217321:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787148442.398396:0:217321:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787148442.398401:0:217321:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787148442.398410:0:217321:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c68fa7200. 00000100:00000001:1.0:1787148442.398417:0:217321:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398430:0:217321:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging adab7261-4d97-43ea-82e8-09637750fa21->MGS req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148442.398442:0:217321:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787148442.398446:0:217321:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398453:0:217321:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f761b4fd] to pc [ptlrpcd_00_01+1] req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398494:0:217321:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398498:0:217321:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787148442.398502:0:217321:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787148442.398505:0:217321:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6f339880. 00000020:00000040:1.0:1787148442.398508:0:217321:0:(genops.c:1231:class_import_get()) import ffff8c5c5776a000 refcount=6 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000100:00000001:1.0:1787148442.398510:0:217321:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787148442.398511:0:217321:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787148442.398513:0:217321:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787148442.398517:0:217321:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c68fa7c00. 00000100:00000001:1.0:1787148442.398520:0:217321:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398525:0:217321:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148442.398532:0:217321:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787148442.398534:0:217321:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398538:0:217321:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000f0312cb5] to pc [ptlrpcd_00_00+0] req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398546:0:217321:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398549:0:217321:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787148442.398572:0:217321:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787148442.398576:0:217321:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6f33aa00. 00000020:00000040:1.0:1787148442.398578:0:217321:0:(genops.c:1231:class_import_get()) import ffff8c5c5776c800 refcount=4 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000100:00000001:1.0:1787148442.398581:0:217321:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787148442.398582:0:217321:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787148442.398584:0:217321:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787148442.398588:0:217321:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c68fa7a00. 00000100:00000001:1.0:1787148442.398591:0:217321:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398596:0:217321:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148442.398603:0:217321:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787148442.398605:0:217321:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398609:0:217321:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003d841376] to pc [ptlrpcd_00_01+1] req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398617:0:217321:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398619:0:217321:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1787148442.398623:0:217321:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1787148442.398625:0:217321:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6f33ad80. 00000020:00000040:1.0:1787148442.398628:0:217321:0:(genops.c:1231:class_import_get()) import ffff8c5c5e0b1800 refcount=4 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000100:00000001:1.0:1787148442.398630:0:217321:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1787148442.398631:0:217321:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787148442.398633:0:217321:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1787148442.398636:0:217321:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c68fa7700. 00000100:00000001:1.0:1787148442.398639:0:217321:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398642:0:217321:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148442.398650:0:217321:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1787148442.398651:0:217321:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398655:0:217321:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b4a6ba03] to pc [ptlrpcd_00_00+0] req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398674:0:217321:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.398676:0:217321:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1787148442.398680:0:217321:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1787148442.398693:0:217321:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12288) 00000100:00000001:1.0:1787148442.398705:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148442.398710:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148442.398713:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787148442.398718:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398726:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148442.398729:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148442.398733:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787148442.398736:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787148442.398739:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148442.398743:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6f339880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731264:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148442.398747:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787148442.398752:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787148442.398755:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148442.398758:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148442.398762:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c47fdcc00. 02000000:00000001:1.0:1787148442.398764:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.398773:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:1.0:1787148442.398778:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c48262790. 00000400:00000010:1.0:1787148442.398786:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026c38. 00000100:00000200:1.0:1787148442.398793:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731264, portal 10 00000100:00000001:1.0:1787148442.398795:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148442.398797:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927363373184 : -127146346178432 : ffff8c5c6f339880) 00000100:00000040:1.0:1787148442.398805:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.398813:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787148442.398822:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:1.0:1787148442.398825:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026440. 00000100:00000001:0.0:1787148442.398828:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000200:1.0:1787148442.398829:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276731264, offset 0 00000100:00000001:0.0:1787148442.398835:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148442.398838:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1787148442.398839:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:1.0:1787148442.398850:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000040:0.0:1787148442.398858:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787148442.398862:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:1.0:1787148442.398871:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1787148442.398872:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787148442.398875:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.398878:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 02000000:00000001:0.0:1787148442.398879:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787148442.398883:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000400:00000200:1.0:1787148442.398885:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 02000000:00000001:0.0:1787148442.398888:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.398891:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000100:00100000:0.0:1787148442.398896:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6f339c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276731136:192.168.202.114@tcp:400:kworker.0 00000400:00000200:1.0:1787148442.398900:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556890:556890:256:4294967295] 192.168.202.114@tcp LPNI seq info [556846:556846:8:4294967295] 00000100:00000001:0.0:1787148442.398903:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787148442.398907:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 00000400:00000200:1.0:1787148442.398910:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 02000000:00000001:0.0:1787148442.398912:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.398916:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787148442.398920:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4b93a000. 00000400:00000200:1.0:1787148442.398923:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 02000000:00000001:0.0:1787148442.398924:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787148442.398934:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000800:00000200:1.0:1787148442.398935:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148442.398941:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f40b0. 00000800:00000010:1.0:1787148442.398943:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7600. 00000400:00000010:0.0:1787148442.398948:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec330. 00000800:00000200:1.0:1787148442.398953:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000100:00000200:0.0:1787148442.398954:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731136, portal 25 00000100:00000001:0.0:1787148442.398959:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787148442.398960:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927363374080 : -127146346177536 : ffff8c5c6f339c00) 00000800:00000200:1.0:1787148442.398965:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000100:00000040:0.0:1787148442.398967:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:1.0:1787148442.398972:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787148442.398980:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787148442.398983:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148442.398987:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fecd48. 00000100:00000001:1.0:1787148442.398988:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787148442.398990:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1873948276731136, offset 0 00000100:00000040:1.0:1787148442.398996:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: using a timeout of 24713 usecs (25 jiffies) 00000400:00000200:0.0:1787148442.398998:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:1.0:1787148442.399004:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148442.399008:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306951193 00000100:00000010:1.0:1787148442.399014:0:2393:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e7832a0. 00000400:00000200:0.0:1787148442.399018:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148442.399023:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000100:00000001:1.0:1787148442.399025:0:2393:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request now active for cpu 0 00000400:00000200:0.0:1787148442.399027:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148442.399032:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148442.399034:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000100:00000001:1.0:1787148442.399037:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306951193 00000400:00000200:0.0:1787148442.399038:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000100:00000010:1.0:1787148442.399043:0:2393:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e7834e0. 00000100:00000001:1.0:1787148442.399048:0:2393:0:(niobuf.c:678:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request now active for cpu 1 00000400:00000200:0.0:1787148442.399052:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556891:556891:255:4294967295] 192.168.202.114@tcp LPNI seq info [556847:556847:7:4294967295] 00000100:00000001:1.0:1787148442.399054:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148442.399059:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399064:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:0.0:1787148442.399064:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148442.399071:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000100:00000040:1.0:1787148442.399079:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000800:00000200:0.0:1787148442.399081:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148442.399089:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474ae700. 00000100:00000001:1.0:1787148442.399091:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148442.399094:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787148442.399097:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 02000000:00000001:1.0:1787148442.399099:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787148442.399130:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787148442.399134:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148442.399137:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6f33ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731520:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148442.399141:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787148442.399143:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787148442.399145:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148442.399147:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148442.399150:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c47fdc400. 02000000:00000001:1.0:1787148442.399153:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787148442.399157:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000400:00000200:1.0:1787148442.399160:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000800:00000200:0.0:1787148442.399163:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474ae700 type 1, nob 320 niov 1 nkiov 1 00000400:00000010:1.0:1787148442.399165:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c48262b00. 00000100:00000001:0.0:1787148442.399170:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:1.0:1787148442.399172:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026770. 00000100:00000200:1.0:1787148442.399177:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731520, portal 4 00000400:00000200:0.0:1787148442.399177:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148442.399181:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148442.399184:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927363378560 : -127146346173056 : ffff8c5c6f33ad80) 00000100:00000001:0.0:1787148442.399184:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 0 is 4306955002 00000100:00000040:1.0:1787148442.399195:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.399195:0:2394:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request active for cpu 0, simply extend its deadline from 4306951193 00000100:00000001:1.0:1787148442.399203:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787148442.399205:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:1.0:1787148442.399208:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f0263b8. 00000100:00000200:1.0:1787148442.399210:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1873948276731520, offset 0 00000100:00000001:0.0:1787148442.399215:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 1 is 4306955002 00000400:00000200:1.0:1787148442.399219:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000100:00000001:0.0:1787148442.399221:0:2394:0:(niobuf.c:685:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request active for cpu 1, simply extend its deadline from 4306951193 00000400:00000200:1.0:1787148442.399226:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:0.0:1787148442.399227:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148442.399232:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.399234:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000100:00000001:0.0:1787148442.399236:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000400:00000200:1.0:1787148442.399239:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787148442.399244:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000100:00000040:0.0:1787148442.399245:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787148442.399248:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000100:00000001:0.0:1787148442.399256:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000400:00000200:1.0:1787148442.399259:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000100:00000001:0.0:1787148442.399260:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.399265:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 00000400:00000200:1.0:1787148442.399266:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556892:556892:254:4294967295] 192.168.202.114@tcp LPNI seq info [556848:556848:6:4294967295] 02000000:00000001:0.0:1787148442.399269:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148442.399274:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.399275:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.399284:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787148442.399291:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000100:00100000:0.0:1787148442.399291:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6f33aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276731392:192.168.202.114@tcp:400:kworker.0 00000800:00000010:1.0:1787148442.399298:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7000. 00000100:00000001:0.0:1787148442.399298:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787148442.399303:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 00000800:00000200:1.0:1787148442.399305:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 02000000:00000001:0.0:1787148442.399307:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1787148442.399312:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 02000000:00000001:0.0:1787148442.399312:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 00000800:00000200:1.0:1787148442.399316:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7000 type 1, nob 320 niov 1 nkiov 1 02000000:00000010:0.0:1787148442.399318:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4b93b200. 00000100:00000001:1.0:1787148442.399320:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.399324:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: using a timeout of 120812 usecs (121 jiffies) 02000000:00000001:0.0:1787148442.399329:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148442.399330:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148442.399334:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306951290 00000100:00000001:1.0:1787148442.399338:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306955002 00000400:00000200:0.0:1787148442.399339:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148442.399341:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306951290 00000400:00000010:0.0:1787148442.399344:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f4790. 00000100:00000001:1.0:1787148442.399345:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306955002 00000100:00000001:1.0:1787148442.399350:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:0.0:1787148442.399351:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec7f8. 00000100:00000001:1.0:1787148442.399352:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:0.0:1787148442.399357:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731392, portal 10 00000100:00000001:1.0:1787148442.399358:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.399363:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148442.399364:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.399366:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927363377664 : -127146346173952 : ffff8c5c6f33aa00) 00000100:00000001:1.0:1787148442.399372:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1787148442.399376:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399377:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:1.0:1787148442.399382:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148442.399384:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148442.399387:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.399389:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:1.0:1787148442.399391:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.399394:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148442.399398:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec990. 00000100:00000200:0.0:1787148442.399402:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276731392, offset 0 00000400:00000200:0.0:1787148442.399407:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148442.399410:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148442.399414:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:0.0:1787148442.399417:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148442.399421:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148442.399423:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787148442.399425:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:0.0:1787148442.399428:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556893:556893:253:4294967295] 192.168.202.114@tcp LPNI seq info [556849:556849:5:4294967295] 00000400:00000200:0.0:1787148442.399431:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148442.399436:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787148442.399440:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148442.399444:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474aed00. 00000800:00000200:0.0:1787148442.399449:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:0.0:1787148442.399454:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:1.0:1787148442.399456:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000200:0.0:1787148442.399457:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474aed00 type 1, nob 320 niov 1 nkiov 1 00000800:00000010:1.0:1787148442.399462:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7600. 00000100:00000001:0.0:1787148442.399463:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.399468:0:2394:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: using a timeout of 16988 usecs (17 jiffies) 00000400:00000200:1.0:1787148442.399470:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148442.399474:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.399479:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148442.399485:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026440 00000100:00000001:0.0:1787148442.399485:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306951186 00000100:00000001:0.0:1787148442.399488:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306955002 00000100:00000001:0.0:1787148442.399491:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306951186 00000100:00000001:0.0:1787148442.399496:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306955002 00000400:00000010:1.0:1787148442.399497:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026440. 00000100:00000001:0.0:1787148442.399501:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148442.399504:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:0.0:1787148442.399504:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.399509:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:1.0:1787148442.399511:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.399513:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.399519:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148442.399522:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1787148442.399525:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148442.399526:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1787148442.399527:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148442.399531:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148442.399532:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.399534:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399545:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399547:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148442.399561:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148442.399563:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474ae700. 00000400:00000200:1.0:1787148442.399566:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.399570:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148442.399573:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fecd48 00000400:00000010:1.0:1787148442.399574:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fecd48. 00000100:00000001:1.0:1787148442.399576:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148442.399580:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399586:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148442.399589:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399595:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399597:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148442.399604:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148442.399606:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7000. 00000400:00000200:1.0:1787148442.399609:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.399612:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148442.399615:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f0263b8 00000400:00000010:1.0:1787148442.399616:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f0263b8. 00000100:00000001:1.0:1787148442.399618:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148442.399622:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399628:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148442.399638:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399644:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399646:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148442.399727:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148442.399730:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474aed00. 00000400:00000200:1.0:1787148442.399733:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.399736:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148442.399739:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec990 00000400:00000010:1.0:1787148442.399740:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec990. 00000100:00000001:1.0:1787148442.399742:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148442.399746:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399753:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148442.399755:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.399761:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148442.399762:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1787148442.402471:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.402500:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148442.402505:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.402508:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.402519:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.402527:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148442.402536:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.402541:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.402546:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.402550:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 4 MB=0x6a858955cae80 00000400:00000200:1.0:1787148442.402555:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.402562:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561df1 [1] + 224 00000800:00000001:1.0:1787148442.402571:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.402578:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.402582:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.402604:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148442.402612:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148442.402622:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148442.402634:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.402662:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148442.402682:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.402694:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403063:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.403068:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148442.403077:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148442.403079:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148442.403084:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c48262b00. 00000400:00000200:0.0:1787148442.403090:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026770 00000400:00000010:0.0:1787148442.403093:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026770. 00000100:00000001:0.0:1787148442.403098:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148442.403153:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148442.403164:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403171:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148442.403173:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403176:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148442.403181:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148442.403184:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148442.403187:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.403191:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148442.403216:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5e0b1c60 time=74 v=36 (36 5 5 5) 00000100:00001000:0.0:1787148442.403232:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5e0b1c18 time=50 v=5 (1 1 1 1) 00000100:00000001:0.0:1787148442.403235:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148442.403237:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148442.403241:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148442.403247:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403254:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148442.403257:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8c5c68fb1800: skip recheck: last_committed 42949695890 00000100:00000001:0.0:1787148442.403259:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148442.403261:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.403267:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403276:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148442.403279:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148442.403282:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148442.403286:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148442.403295:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6f33ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731520:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148442.403301:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403305:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148442.403308:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6f33ad80 x1873948276731520/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403316:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148442.403320:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148442.403324:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c47fdc400. 02000000:00000001:0.0:1787148442.403330:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148442.403333:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148442.403335:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5e0b1800 refcount=3 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148442.403338:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148442.403341:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c68fa7700. 02000000:00000001:0.0:1787148442.403345:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148442.403347:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148442.403349:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6f33ad80. 00000100:00000001:0.0:1787148442.403362:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148442.403363:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403367:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403369:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000800:00000001:1.0:1787148442.403370:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000100:00000001:0.0:1787148442.403372:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148442.403375:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403377:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1787148442.403381:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148442.403385:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.403387:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.403391:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.403398:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148442.403403:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.403407:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.403410:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.403414:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 25 MB=0x6a858955cad00 00000400:00000200:1.0:1787148442.403418:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.403423:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561de1 [1] + 224 00000800:00000001:1.0:1787148442.403427:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.403433:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.403436:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.403440:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148442.403444:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148442.403450:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148442.403460:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.403480:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148442.403493:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.403495:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403570:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.403574:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148442.403578:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148442.403580:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148442.403582:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f40b0. 00000400:00000200:0.0:1787148442.403586:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec330 00000400:00000010:0.0:1787148442.403588:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec330. 00000100:00000001:0.0:1787148442.403590:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148442.403595:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148442.403605:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403611:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148442.403613:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403615:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148442.403617:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148442.403619:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148442.403620:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.403630:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148442.403634:0:2394:0:(import.c:2149:obd_at_measure()) add 5 to ffff8c5c5776b460 time=74 v=5 (5 5 5 5) 00000100:00001000:0.0:1787148442.403637:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776b418 time=126 v=5 (1 1 0 0) 00000100:00000001:0.0:1787148442.403639:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148442.403641:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148442.403643:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148442.403645:0:2394:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403647:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.403651:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403657:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148442.403665:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148442.403666:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148442.403671:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148442.403678:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6f339c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276731136:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148442.403694:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403697:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148442.403700:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6f339c00 x1873948276731136/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148458 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.403706:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148442.403708:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148442.403710:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4b93a000. 02000000:00000001:0.0:1787148442.403715:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148442.403716:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148442.403717:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776b000 refcount=3 obd=MGC192.168.202.114@tcp 00000020:00000001:0.0:1787148442.403719:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148442.403721:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c68fa7200. 02000000:00000001:0.0:1787148442.403723:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148442.403724:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148442.403725:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6f339c00. 00000100:00000001:0.0:1787148442.403729:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148442.403730:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.403732:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403734:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.403736:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148442.403737:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.403739:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1787148442.406370:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.406391:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148442.406396:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.406398:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.406410:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.406420:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148442.406431:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.406436:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.406443:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.406447:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cad80 00000400:00000200:1.0:1787148442.406452:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.406458:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561dd1 [1] + 224 00000800:00000001:1.0:1787148442.406467:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.406476:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.406479:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.406483:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148442.406493:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148442.406504:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148442.406517:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.406543:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148442.406553:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.406558:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.407912:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.407921:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148442.407931:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148442.407934:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148442.407939:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c48262790. 00000400:00000200:0.0:1787148442.407949:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026c38 00000400:00000010:0.0:1787148442.407952:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026c38. 00000100:00000001:0.0:1787148442.407958:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148442.407993:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148442.408075:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.408085:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148442.408088:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.408091:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148442.408095:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148442.408097:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148442.408101:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.408160:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148442.408173:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776a460 time=74 v=36 (36 36 36 36) 00000100:00001000:0.0:1787148442.408179:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776a418 time=126 v=5 (1 1 0 0) 00000100:00000001:0.0:1787148442.408183:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148442.408184:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148442.408187:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148442.408193:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.408199:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148442.408201:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8c5c68fb1800: skip recheck: last_committed 30064771460 00000100:00000001:0.0:1787148442.408204:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148442.408206:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.408212:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.408220:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148442.408224:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148442.408226:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148442.408230:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148442.408239:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6f339880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731264:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148442.408246:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.408249:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148442.408260:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6f339880 x1873948276731264/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.408267:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148442.408270:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148442.408275:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c47fdcc00. 02000000:00000001:0.0:1787148442.408282:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148442.408625:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148442.408629:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5776a000 refcount=5 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148442.408632:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148442.408635:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c68fa7c00. 02000000:00000001:0.0:1787148442.408640:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148442.408643:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148442.408646:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6f339880. 00000100:00000001:0.0:1787148442.408650:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148442.408651:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.408654:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.408661:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148442.408662:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.408664:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.408667:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1787148442.417242:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.417264:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148442.417268:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.417271:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.417280:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.417289:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148442.417298:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.417318:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.417324:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.417328:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cae00 00000400:00000200:1.0:1787148442.417333:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148442.417339:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e01 [1] + 224 00000800:00000001:1.0:1787148442.417347:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.417354:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148442.417358:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148442.417362:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148442.417369:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148442.417378:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148442.417389:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148442.417414:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148442.417423:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148442.417426:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.417506:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.417511:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148442.417517:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148442.417519:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148442.417523:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f4790. 00000400:00000200:0.0:1787148442.417528:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec7f8 00000400:00000010:0.0:1787148442.417530:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec7f8. 00000100:00000001:0.0:1787148442.417534:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148442.417538:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148442.417547:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.417554:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148442.417556:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.417558:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148442.417561:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148442.417570:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148442.417572:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148442.417575:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148442.417585:0:2394:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776cc60 time=74 v=36 (36 36 36 36) 00000100:00001000:0.0:1787148442.417590:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776cc18 time=101 v=5 (1 1 0 0) 00000100:00000001:0.0:1787148442.417592:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148442.417594:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148442.417597:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1787148442.417602:0:2394:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1787148442.417610:0:2394:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.417614:0:2394:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148442.417616:0:2394:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8c5c68fb1800: skip recheck: last_committed 21474836540 00000100:00000001:0.0:1787148442.417618:0:2394:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148442.417619:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148442.417624:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.417630:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148442.417633:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148442.417636:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148442.417639:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148442.417647:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6f33aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276731392:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148442.417652:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.417655:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148442.417658:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6f33aa00 x1873948276731392/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148497 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148442.417665:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148442.417667:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148442.417670:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4b93b200. 02000000:00000001:0.0:1787148442.417761:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148442.417770:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148442.417772:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776c800 refcount=3 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148442.417775:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148442.417778:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c68fa7a00. 02000000:00000001:0.0:1787148442.417781:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148442.417783:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148442.417785:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6f33aa00. 00000100:00000001:0.0:1787148442.417788:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148442.417789:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.417791:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148442.417799:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148442.417801:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148442.417803:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148442.417806:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148442.590259:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148442.590268:0:256195:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1787148442.590270:0:256195:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1787148442.590273:0:256195:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148442.590279:0:256195:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1787148442.590282:0:256195:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148442.590284:0:256195:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148442.590286:0:256195:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148442.590291:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148442.590292:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148442.590386:0:244719:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148442.590390:0:244719:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148442.590393:0:244719:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148443.614622:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148443.614631:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148443.614640:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148443.614641:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148443.614773:0:244722:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148443.614798:0:244722:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148443.614802:0:244722:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148444.638329:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148444.638337:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148444.638343:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148444.638368:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787148444.638540:0:256812:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787148444.638544:0:256812:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787148444.638548:0:256812:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148445.664610:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148445.664619:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148445.664626:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148445.664627:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148445.675401:0:235348:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148445.675410:0:235348:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148445.675416:0:235348:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787148446.238324:0:231971:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342000 (cpu 0) has reached its deadline 4306955002, at 4306955008 00000100:00000010:0.0:1787148446.238341:0:231971:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e7832a0. 00000100:00000040:1.0:1787148446.244322:0:256195:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342110 (cpu 1) has reached its deadline 4306955002, at 4306955014 00000100:00000010:1.0:1787148446.244337:0:256195:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e7834e0. 00000080:00000001:2.0:1787148446.300008:0:277668:0:(llite_lib.c:4245:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148446.686177:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148446.686184:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148446.686190:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148446.686192:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787148446.686213:0:235349:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787148446.686215:0:235349:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787148446.686219:0:235349:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1787148447.519376:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) adab7261-4d97-43ea-82e8-09637750fa21->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148447.519388:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148447.519397:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfeb100. 00000020:00000040:0.0:1787148447.519403:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776b000 refcount=4 obd=MGC192.168.202.114@tcp 00000100:00000001:0.0:1787148447.519407:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148447.519410:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148447.519415:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148447.519424:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474ae100. 00000100:00000001:0.0:1787148447.519431:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.519441:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging adab7261-4d97-43ea-82e8-09637750fa21->MGS req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787148447.519452:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148447.519457:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.519463:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005dc7efec] to pc [ptlrpcd_00_01+1] req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.519501:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.519504:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148447.519507:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148447.519510:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfea300. 00000020:00000040:0.0:1787148447.519512:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776a000 refcount=6 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000100:00000001:0.0:1787148447.519515:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148447.519516:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148447.519518:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148447.519521:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474ae000. 00000100:00000001:0.0:1787148447.519525:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.519530:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148447.519629:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.519635:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148447.519647:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787148447.519654:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.519662:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148447.519664:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.519668:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787148447.519671:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787148447.519673:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148447.519677:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276731648:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148447.519682:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787148447.519688:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787148447.519690:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.519693:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148447.519698:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c46ee7800. 02000000:00000001:1.0:1787148447.519699:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148447.519706:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:1.0:1787148447.519711:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c482624d0. 00000400:00000010:1.0:1787148447.519718:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f0263b8. 00000100:00000200:1.0:1787148447.519725:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731648, portal 25 00000100:00000001:1.0:1787148447.519734:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148447.519735:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343128832 : -127146366422784 : ffff8c5c6dfeb100) 00000100:00000040:1.0:1787148447.519742:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.519749:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787148447.519751:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:1.0:1787148447.519755:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026440. 00000100:00000200:1.0:1787148447.519758:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1873948276731648, offset 0 00000400:00000200:1.0:1787148447.519765:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:1.0:1787148447.519774:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.519781:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:1.0:1787148447.519787:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787148447.519792:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:1.0:1787148447.519796:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787148447.519798:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:1.0:1787148447.519802:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556894:556894:256:4294967295] 192.168.202.114@tcp LPNI seq info [556850:556850:8:4294967295] 00000400:00000200:1.0:1787148447.519806:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.519815:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787148447.519821:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:1.0:1787148447.519826:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7a00. 00000800:00000200:1.0:1787148447.519833:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:1.0:1787148447.519841:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:1.0:1787148447.519847:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787148447.519858:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148447.519862:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148447.519865:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 0 is 4306960122 00000100:00000010:1.0:1787148447.519878:0:2394:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e7834e0. 00000100:00000001:0.0:1787148447.520376:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148447.520380:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520387:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1309e01] to pc [ptlrpcd_00_00+0] req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.520398:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520401:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148447.520405:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148447.520409:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfeb800. 00000020:00000040:0.0:1787148447.520412:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776c800 refcount=4 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000100:00000001:0.0:1787148447.520415:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148447.520417:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148447.520419:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148447.520423:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474aeb00. 00000100:00000001:0.0:1787148447.520426:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520431:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787148447.520437:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148447.520447:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520452:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000007bc9d663] to pc [ptlrpcd_00_01+1] req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.520460:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520462:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148447.520466:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148447.520468:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfeaa00. 00000020:00000040:0.0:1787148447.520471:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5e0b1800 refcount=4 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000100:00000001:0.0:1787148447.520473:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148447.520474:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148447.520476:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148447.520482:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474aed00. 00000100:00000001:0.0:1787148447.520484:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520488:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787148447.520494:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148447.520496:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520500:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000882e9706] to pc [ptlrpcd_00_00+0] req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.520507:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.520509:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:0.0:1787148447.520512:0:232905:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:0.0:1787148447.520518:0:232905:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12293) 00000100:00000001:0.0:1787148447.522506:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148447.522514:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148447.522517:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787148447.522530:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.522852:0:2394:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1787148447.522877:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 1 is 4306960125 00000100:00000001:0.0:1787148447.523672:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787148447.523674:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.523702:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787148447.523706:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148447.523710:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787148447.523716:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731776:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148447.523720:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787148447.523727:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787148447.523730:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.523733:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787148447.523743:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4a665e00. 02000000:00000001:0.0:1787148447.523745:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787148447.523754:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:0.0:1787148447.523761:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f4790. 00000400:00000010:0.0:1787148447.523769:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec7f8. 00000100:00000200:0.0:1787148447.523779:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731776, portal 10 00000100:00000001:0.0:1787148447.523782:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787148447.523784:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343125248 : -127146366426368 : ffff8c5c6dfea300) 00000100:00000040:0.0:1787148447.523795:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.523803:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787148447.523806:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148447.523809:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec330. 00000100:00000200:0.0:1787148447.523813:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276731776, offset 0 00000400:00000200:0.0:1787148447.523821:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148447.523831:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148447.523840:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:0.0:1787148447.523846:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148447.523852:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148447.523856:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787148447.523859:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:0.0:1787148447.523863:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556895:556895:255:4294967295] 192.168.202.114@tcp LPNI seq info [556851:556851:7:4294967295] 00000400:00000200:0.0:1787148447.523877:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000010:1.0:1787148447.524145:0:2394:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e7836c0. 00000100:00000001:1.0:1787148447.524160:0:2394:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1787148447.524169:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148447.524173:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.524177:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.524180:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.524191:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148447.524194:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787148447.524197:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.524199:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148447.524200:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787148447.524206:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.524213:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148447.524215:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.524217:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787148447.524219:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787148447.524221:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148447.524224:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfeb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276731904:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148447.524227:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787148447.524230:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787148447.525618:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.525628:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148447.525635:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c46ee6600. 02000000:00000001:1.0:1787148447.525650:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148447.525658:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:1.0:1787148447.525663:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c48262f78. 00000400:00000010:1.0:1787148447.525670:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026000. 00000100:00000200:1.0:1787148447.525677:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276731904, portal 10 00000100:00000001:1.0:1787148447.525679:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148447.525681:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343130624 : -127146366420992 : ffff8c5c6dfeb800) 00000100:00000040:1.0:1787148447.525692:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.525700:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787148447.525712:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:1.0:1787148447.525715:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026198. 00000100:00000200:1.0:1787148447.525718:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276731904, offset 0 00000400:00000200:1.0:1787148447.525724:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148447.526802:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787148447.526831:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148447.526837:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474ae700. 00000800:00000200:0.0:1787148447.526850:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:0.0:1787148447.526859:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:0.0:1787148447.526863:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474ae700 type 1, nob 320 niov 1 nkiov 1 00000400:00000200:1.0:1787148447.527336:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.527346:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:1.0:1787148447.527351:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787148447.527355:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:1.0:1787148447.527358:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787148447.527361:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:1.0:1787148447.527365:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556896:556896:254:4294967295] 192.168.202.114@tcp LPNI seq info [556852:556852:6:4294967295] 00000400:00000200:1.0:1787148447.527369:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.527376:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787148447.527381:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:1.0:1787148447.527387:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7600. 00000800:00000200:1.0:1787148447.527391:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:1.0:1787148447.527395:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:1.0:1787148447.527398:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787148447.527402:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148447.527408:0:2394:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: using a timeout of 16988 usecs (17 jiffies) 00000400:00000200:1.0:1787148447.527411:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148447.527414:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306956313 00000100:00000001:1.0:1787148447.527417:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306960122 00000100:00000001:1.0:1787148447.527420:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306956313 00000100:00000001:1.0:1787148447.527422:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306960125 00000100:00000001:1.0:1787148447.527424:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148447.527428:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.527433:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.527437:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.527447:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148447.527457:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787148447.527460:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.527462:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148447.527464:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.527466:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1787148447.527518:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148447.527523:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7a00. 00000400:00000200:1.0:1787148447.527529:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.527534:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:0.0:1787148447.528966:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.528978:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: using a timeout of 24708 usecs (25 jiffies) 00000400:00000200:0.0:1787148447.528985:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:0.0:1787148447.528989:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306956323 00000100:00000001:0.0:1787148447.528992:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306960122 00000100:00000001:0.0:1787148447.528995:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306956323 00000100:00000001:0.0:1787148447.528997:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306960125 00000100:00000001:0.0:1787148447.529000:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148447.529003:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.529006:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787148447.529017:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.529027:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787148447.529029:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.529033:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787148447.529037:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148447.529040:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787148447.529044:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfeaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732032:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148447.529048:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787148447.529054:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787148447.529058:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.529061:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787148447.529067:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4a665600. 02000000:00000001:0.0:1787148447.529069:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787148447.529073:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:0.0:1787148447.529086:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f40b0. 00000400:00000010:0.0:1787148447.529095:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec6e8. 00000100:00000200:0.0:1787148447.529128:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276732032, portal 4 00000100:00000001:0.0:1787148447.529131:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787148447.529133:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343127040 : -127146366424576 : ffff8c5c6dfeaa00) 00000100:00000040:0.0:1787148447.529142:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.529149:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787148447.529152:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148447.529155:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec110. 00000100:00000200:0.0:1787148447.529159:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1873948276732032, offset 0 00000400:00000200:0.0:1787148447.529165:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148447.529174:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148447.529181:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:0.0:1787148447.529185:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148447.529189:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148447.529192:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787148447.529195:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:0.0:1787148447.529198:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556897:556897:253:4294967295] 192.168.202.114@tcp LPNI seq info [556853:556853:5:4294967295] 00000400:00000200:0.0:1787148447.529203:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148447.529212:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787148447.529218:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148447.529222:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474ae300. 00000800:00000200:0.0:1787148447.529229:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:0.0:1787148447.529235:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:0.0:1787148447.529240:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474ae300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787148447.529244:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.529247:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: using a timeout of 120721 usecs (121 jiffies) 00000400:00000200:0.0:1787148447.529251:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:0.0:1787148447.529253:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306956420 00000100:00000001:0.0:1787148447.529255:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306960122 00000100:00000001:0.0:1787148447.529262:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306956420 00000100:00000001:0.0:1787148447.529264:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306960125 00000100:00000001:0.0:1787148447.529266:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148447.529268:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.529271:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.529274:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.529283:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000400:00000200:1.0:1787148447.529347:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026440 00000400:00000010:1.0:1787148447.529350:0:2386:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026440. 00000100:00000001:1.0:1787148447.529355:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148447.529360:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529371:0:2386:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.529374:0:2386:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529380:0:2386:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.529382:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148447.529410:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148447.529414:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474ae700. 00000400:00000200:1.0:1787148447.529419:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.529423:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148447.529426:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec330 00000400:00000010:1.0:1787148447.529427:0:2386:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec330. 00000100:00000001:1.0:1787148447.529430:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148447.529434:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529441:0:2386:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.529444:0:2386:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529450:0:2386:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.529451:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148447.529468:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148447.529470:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7600. 00000400:00000200:1.0:1787148447.529473:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.529476:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148447.529479:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026198 00000400:00000010:1.0:1787148447.529481:0:2386:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026198. 00000100:00000001:1.0:1787148447.529484:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148447.529487:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529494:0:2386:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.529497:0:2386:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.529502:0:2386:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.529504:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1787148447.531153:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1787148447.531163:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474ae300. 00000400:00000200:1.0:1787148447.531171:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.531178:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:1.0:1787148447.531184:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec110 00000400:00000010:1.0:1787148447.531185:0:2386:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec110. 00000100:00000001:1.0:1787148447.531190:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1787148447.531196:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.531206:0:2386:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.531209:0:2386:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.531216:0:2386:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.531217:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000001:0.0:1787148447.531383:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:0.0:1787148447.531389:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148447.531392:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148447.531394:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.531397:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1787148447.540195:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.540214:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148447.540219:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.540222:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.540231:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.540253:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148447.540263:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.540269:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.540274:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.540278:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955caf80 00000400:00000200:1.0:1787148447.540284:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.540290:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e21 [1] + 224 00000800:00000001:1.0:1787148447.540297:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.540302:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.540305:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.540310:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148447.540318:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.540328:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148447.540341:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.540394:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148447.540400:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148447.541093:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148447.541133:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.541139:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148447.541142:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.541145:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.541150:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.541155:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148447.541160:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.541163:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.541166:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.541169:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cb000 00000400:00000200:1.0:1787148447.541173:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.541178:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e31 [1] + 224 00000800:00000001:1.0:1787148447.541183:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.541191:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.541200:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.541204:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148447.541209:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.541215:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148447.541224:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.541239:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148447.541243:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.541246:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.541260:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.541262:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787148447.541268:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787148447.541271:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787148447.541274:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c48262f78. 00000400:00000200:1.0:1787148447.541278:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026000 00000400:00000010:1.0:1787148447.541281:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026000. 00000100:00000001:1.0:1787148447.541284:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.541288:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787148447.541298:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.541304:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787148447.541305:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.541308:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1787148447.541311:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787148447.541314:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787148447.541316:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.541319:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787148447.541328:0:2394:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776cc60 time=79 v=36 (36 36 36 36) 00000100:00001000:1.0:1787148447.541333:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776cc18 time=106 v=5 (1 1 0 0) 00000100:00000001:1.0:1787148447.541336:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787148447.541338:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148447.541341:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1787148447.541352:0:2394:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1787148447.541359:0:2394:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.541362:0:2394:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787148447.541366:0:2394:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8c5c68fb1800: skip recheck: last_committed 21474836540 00000100:00000001:1.0:1787148447.541368:0:2394:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787148447.541369:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148447.541373:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.541380:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148447.541382:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787148447.541384:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787148447.541388:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787148447.541396:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfeb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276731904:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148447.541403:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.541405:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.541409:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfeb800 x1873948276731904/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.541416:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787148447.541418:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787148447.541422:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c46ee6600. 02000000:00000001:1.0:1787148447.541425:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787148447.541428:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1787148447.541430:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776c800 refcount=3 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000020:00000001:1.0:1787148447.541432:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1787148447.541435:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474aeb00. 02000000:00000001:1.0:1787148447.541440:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787148447.541442:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787148447.541443:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfeb800. 00000100:00000001:1.0:1787148447.541446:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787148447.541455:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.541458:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.541460:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.541462:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148447.541463:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.541465:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1787148447.547185:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547210:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148447.547215:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547218:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.547229:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547237:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148447.547249:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547255:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547262:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547267:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 25 MB=0x6a858955caf00 00000400:00000200:1.0:1787148447.547273:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547280:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e11 [1] + 224 00000800:00000001:1.0:1787148447.547288:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547293:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.547297:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.547302:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148447.547310:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.547319:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148447.547332:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.547353:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148447.547365:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547368:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1787148447.547370:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547372:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.547375:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547379:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:1.0:1787148447.547401:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547404:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547406:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547409:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 4 MB=0x6a858955cb080 00000400:00000200:1.0:1787148447.547413:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148447.547417:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e41 [1] + 224 00000800:00000001:1.0:1787148447.547421:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547431:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1787148447.547433:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1787148447.547436:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:1.0:1787148447.547440:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.547444:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148447.547452:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.547459:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1787148447.547463:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1787148447.547466:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.547481:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.547485:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787148447.547491:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787148447.547493:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787148447.547496:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c482624d0. 00000400:00000200:1.0:1787148447.547500:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f0263b8 00000400:00000010:1.0:1787148447.547502:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f0263b8. 00000100:00000001:1.0:1787148447.547505:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148447.547510:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787148447.547518:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.547524:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787148447.547525:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.547547:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1787148447.547550:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787148447.547552:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787148447.547555:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148447.547558:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787148447.547566:0:2394:0:(import.c:2149:obd_at_measure()) add 5 to ffff8c5c5776b460 time=79 v=5 (5 5 5 5) 00000100:00001000:1.0:1787148447.547570:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776b418 time=131 v=5 (1 1 0 0) 00000100:00000001:1.0:1787148447.547573:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787148447.547574:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148447.547577:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787148447.547579:0:2394:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.547582:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148447.547588:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.547595:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148447.547598:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787148447.547600:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787148447.547604:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787148447.547613:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276731648:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148447.547618:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.547620:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148447.547623:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfeb100 x1873948276731648/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148463 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148447.547630:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787148447.547632:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787148447.547646:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c46ee7800. 02000000:00000001:1.0:1787148447.547651:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787148447.547653:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1787148447.547655:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776b000 refcount=3 obd=MGC192.168.202.114@tcp 00000020:00000001:1.0:1787148447.547657:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1787148447.547660:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474ae100. 02000000:00000001:1.0:1787148447.547664:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787148447.547671:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787148447.547674:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfeb100. 00000100:00000001:1.0:1787148447.547677:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787148447.547679:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.547681:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148447.547688:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148447.547690:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148447.547692:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148447.547694:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:0.0:1787148447.549394:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148447.549401:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148447.549405:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f4790. 00000400:00000200:0.0:1787148447.549414:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec7f8 00000400:00000010:0.0:1787148447.549416:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec7f8. 00000100:00000001:0.0:1787148447.549422:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148447.549430:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148447.549440:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549447:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148447.549449:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549451:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148447.549454:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148447.549456:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148447.549458:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.549461:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148447.549470:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776a460 time=79 v=36 (36 36 36 36) 00000100:00001000:0.0:1787148447.549474:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776a418 time=131 v=5 (1 1 0 0) 00000100:00000001:0.0:1787148447.549477:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148447.549478:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148447.549495:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148447.549499:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.549504:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148447.549506:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8c5c68fb1800: skip recheck: last_committed 30064771460 00000100:00000001:0.0:1787148447.549508:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148447.549509:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.549515:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549521:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148447.549524:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148447.549525:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148447.549528:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148447.549547:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276731776:192.168.202.114@tcp:400:kworker.0 00000400:00000001:0.0:1787148447.549554:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148447.549555:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148447.549557:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f40b0. 00000400:00000200:0.0:1787148447.549560:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec6e8 00000400:00000010:0.0:1787148447.549561:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec6e8. 00000100:00000001:0.0:1787148447.549564:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148447.549568:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148447.549577:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549583:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148447.549584:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549586:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148447.549587:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148447.549588:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148447.549590:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148447.549592:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148447.549596:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5e0b1c60 time=79 v=36 (36 5 5 5) 00000100:00001000:0.0:1787148447.549605:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5e0b1c18 time=55 v=5 (1 1 1 1) 00000100:00000001:0.0:1787148447.549607:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148447.549608:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148447.549610:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148447.549612:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.549614:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148447.549615:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8c5c68fb1800: skip recheck: last_committed 42949695890 00000100:00000001:0.0:1787148447.549616:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148447.549618:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148447.549622:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549627:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148447.549629:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148447.549630:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148447.549634:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148447.549652:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfeaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732032:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148447.549657:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549660:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148447.549664:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfea300 x1873948276731776/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549670:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148447.549673:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148447.549676:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4a665e00. 02000000:00000001:0.0:1787148447.549680:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148447.549682:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148447.549684:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5776a000 refcount=5 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148447.549687:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148447.549689:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474ae000. 02000000:00000001:0.0:1787148447.549694:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148447.549695:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148447.549697:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfea300. 00000100:00000001:0.0:1787148447.549705:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148447.549707:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549708:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148447.549712:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfeaa00 x1873948276732032/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148502 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148447.549719:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148447.549720:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148447.549722:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4a665600. 02000000:00000001:0.0:1787148447.549724:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148447.549726:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148447.549727:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5e0b1800 refcount=3 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148447.549729:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148447.549731:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474aed00. 02000000:00000001:0.0:1787148447.549733:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148447.549734:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148447.549735:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfeaa00. 00000100:00000001:0.0:1787148447.549737:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148447.549738:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549741:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148447.549749:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148447.549751:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148447.549754:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148447.549756:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148447.710202:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148447.710209:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148447.710214:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148447.710216:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148447.710358:0:244719:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148447.710362:0:244719:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148447.710365:0:244719:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148448.734162:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148448.734170:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148448.734175:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148448.734176:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148448.734308:0:244722:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148448.734323:0:244722:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148448.734327:0:244722:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148449.758240:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148449.758246:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148449.758252:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148449.758254:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787148449.773870:0:256812:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787148449.773876:0:256812:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787148449.773880:0:256812:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148450.782169:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148450.782175:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148450.782181:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148450.782182:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148450.782356:0:235348:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148450.782360:0:235348:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148450.782363:0:235348:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1787148451.358186:0:256195:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342110 (cpu 1) has reached its deadline 4306960125, at 4306960128 00000100:00000040:0.0:1787148451.358561:0:231971:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342000 (cpu 0) has reached its deadline 4306960122, at 4306960128 00000100:00000010:0.0:1787148451.358573:0:231971:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e7834e0. 00000100:00000010:1.0:1787148451.361807:0:256195:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e7836c0. 00010000:00000001:1.0:1787148451.806193:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148451.806201:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148451.806206:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148451.806210:0:256195:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1787148451.806212:0:256195:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1787148451.806220:0:256195:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:1.0:1787148451.806222:0:256195:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:1.0:1787148451.806223:0:256195:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148451.806226:0:256195:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1787148451.806229:0:256195:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148451.806230:0:256195:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148451.806243:0:256195:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787148451.806269:0:235349:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787148451.806271:0:235349:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787148451.806274:0:235349:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000800:00000200:1.0:1787148452.000861:0:2384:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4992e3c0] -> 12345-192.168.202.14@tcp (6) 00000800:00000200:1.0:1787148452.000870:0:2384:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.14@tcp ip 192.168.202.14:988 00000800:00000200:1.0:1787148452.000876:0:2384:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c60f28c00 type 192, nob 24 niov 1 nkiov 0 00000800:00000200:0.0:1787148452.001195:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000001:0.0:1787148452.001201:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.001216:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1787148452.001221:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.001224:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148452.639031:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) adab7261-4d97-43ea-82e8-09637750fa21->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148452.639043:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148452.639054:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfeaa00. 00000020:00000040:0.0:1787148452.639074:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776b000 refcount=4 obd=MGC192.168.202.114@tcp 00000100:00000001:0.0:1787148452.639078:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148452.639082:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148452.639087:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148452.639095:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474ae600. 00000100:00000001:0.0:1787148452.639131:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639144:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging adab7261-4d97-43ea-82e8-09637750fa21->MGS req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787148452.639155:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148452.639159:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639165:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000882e9706] to pc [ptlrpcd_00_01+1] req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.639185:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639188:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148452.639192:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148452.639195:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfea300. 00000020:00000040:0.0:1787148452.639197:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776a000 refcount=6 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000100:00000001:0.0:1787148452.639199:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148452.639200:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148452.639202:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148452.639205:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474aee00. 00000100:00000001:0.0:1787148452.639208:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639212:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0000_UUID req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1787148452.639219:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148452.639221:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639225:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1309e01] to pc [ptlrpcd_00_00+0] req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.639233:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639236:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148452.639249:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1787148452.639252:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfe9180. 00000020:00000040:0.0:1787148452.639255:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5776c800 refcount=4 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000100:00000001:0.0:1787148452.639257:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148452.639259:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1787148452.639260:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148452.639264:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474aed00. 00000100:00000001:0.0:1787148452.639267:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639271:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-MDT0001_UUID req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148452.639279:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.639279:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148452.639281:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.639283:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000040:0.0:1787148452.639285:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000004313bd28] to pc [ptlrpcd_00_01+1] req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.639286:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787148452.639292:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.639292:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639295:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1787148452.639299:0:232905:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1787148452.639300:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148452.639303:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000010:0.0:1787148452.639304:0:232905:0:(client.c:593:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff8c5c6dfeb100. 00000020:00000040:0.0:1787148452.639306:0:232905:0:(genops.c:1231:class_import_get()) import ffff8c5c5e0b1800 refcount=4 obd=lustre-OST0000-osc-ffff8c5c68fb1800 02000000:00000001:1.0:1787148452.639307:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:0.0:1787148452.639308:0:232905:0:(client.c:845:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1787148452.639309:0:232905:0:(sec.c:439:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1787148452.639310:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148452.639310:0:232905:0:(sec.c:463:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.639313:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1787148452.639313:0:232905:0:(sec_null.c:135:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff8c5c474ae000. 00000100:00000001:0.0:1787148452.639316:0:232905:0:(client.c:925:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148452.639317:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfeaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276732160:192.168.202.114@tcp:400:kworker.0 00000100:00000040:0.0:1787148452.639319:0:232905:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0000_UUID req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1787148452.639322:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1787148452.639325:0:232905:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1787148452.639327:0:232905:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.639330:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 00000100:00000040:0.0:1787148452.639331:0:232905:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005dc7efec] to pc [ptlrpcd_00_00+0] req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 02000000:00000001:1.0:1787148452.639333:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.639336:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148452.639341:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c46ee7800. 02000000:00000001:1.0:1787148452.639343:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.639348:0:232905:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.639350:0:232905:0:(pinger.c:229:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000400:00000200:1.0:1787148452.639353:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000040:0.0:1787148452.639354:0:232905:0:(pinger.c:247:ptlrpc_pinger_process_import()) 7e31181a-25e1-4c2b-a23d-b0f7747006f1->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000400:00000010:1.0:1787148452.639358:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c482624d0. 00000100:00000040:0.0:1787148452.639359:0:232905:0:(pinger.c:304:ptlrpc_pinger_main()) next wakeup in 5 (12299) 00000400:00000010:1.0:1787148452.639367:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f0263b8. 00000100:00000001:0.0:1787148452.639372:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.639374:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000200:1.0:1787148452.639375:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276732160, portal 25 00000100:00000001:0.0:1787148452.639375:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000001:1.0:1787148452.639377:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148452.639379:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343127040 : -127146366424576 : ffff8c5c6dfeaa00) 00000100:00000040:0.0:1787148452.639380:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1787148452.639385:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.639387:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787148452.639388:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.639390:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 00000100:00000001:1.0:1787148452.639392:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 02000000:00000001:0.0:1787148452.639392:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148452.639394:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787148452.639397:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732288:192.168.202.114@tcp:400:kworker.0 00000100:00000040:1.0:1787148452.639399:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000100:00000001:0.0:1787148452.639400:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 00000400:00000010:1.0:1787148452.639402:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026000. 02000000:00000001:0.0:1787148452.639402:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787148452.639403:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.639405:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 00000100:00000200:1.0:1787148452.639406:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1873948276732160, offset 0 02000000:00000010:0.0:1787148452.639408:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4a665e00. 02000000:00000001:0.0:1787148452.639409:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148452.639412:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148452.639413:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:0.0:1787148452.639416:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f40b0. 00000400:00000010:0.0:1787148452.639419:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec6e8. 00000400:00000200:1.0:1787148452.639422:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000200:0.0:1787148452.639422:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276732288, portal 10 00000100:00000001:0.0:1787148452.639423:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787148452.639425:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343125248 : -127146366426368 : ffff8c5c6dfea300) 00000400:00000200:1.0:1787148452.639430:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000100:00000040:0.0:1787148452.639432:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1787148452.639436:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:0.0:1787148452.639438:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000400:00000200:1.0:1787148452.639440:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000100:00000040:0.0:1787148452.639443:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000200:1.0:1787148452.639445:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000010:0.0:1787148452.639446:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec7f8. 00000400:00000200:1.0:1787148452.639448:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000100:00000200:0.0:1787148452.639448:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276732288, offset 0 00000400:00000200:0.0:1787148452.639452:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:1.0:1787148452.640016:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556898:556898:256:4294967295] 192.168.202.114@tcp LPNI seq info [556854:556854:8:4294967295] 00000400:00000200:1.0:1787148452.640024:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148452.640036:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787148452.640043:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:1.0:1787148452.640049:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7600. 00000800:00000200:1.0:1787148452.640056:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:1.0:1787148452.640063:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:1.0:1787148452.640067:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7600 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787148452.640084:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148452.640089:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148452.640093:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 0 is 4306965242 00000100:00000010:1.0:1787148452.640097:0:2394:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e7836c0. 00000100:00000001:1.0:1787148452.640128:0:2394:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1787148452.640136:0:2394:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS new deadline estimation for cpu 1 is 4306965242 00000100:00000010:1.0:1787148452.640140:0:2394:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff8c5c5e783840. 00000100:00000001:1.0:1787148452.640143:0:2394:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.202.114@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1787148452.640146:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148452.640150:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.640152:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.640154:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.640162:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148452.640172:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1787148452.640174:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148452.640175:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148452.640177:0:2394:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1787148452.640182:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.640262:0:2394:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1787148452.640264:0:2394:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.640267:0:2394:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1787148452.640269:0:2394:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1787148452.640271:0:2394:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1787148452.640275:0:2394:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfe9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276732416:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148452.640278:0:2394:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1787148452.640282:0:2394:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1787148452.640284:0:2394:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.640285:0:2394:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1787148452.640289:0:2394:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c46ee6600. 02000000:00000001:1.0:1787148452.640290:0:2394:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148452.640294:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:1.0:1787148452.640298:0:2394:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c48262f78. 00000400:00000010:1.0:1787148452.640302:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026198. 00000100:00000200:1.0:1787148452.640306:0:2394:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276732416, portal 10 00000100:00000001:1.0:1787148452.640308:0:2394:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1787148452.640309:0:2394:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343120768 : -127146366430848 : ffff8c5c6dfe9180) 00000100:00000040:1.0:1787148452.640316:0:2394:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.640322:0:2394:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1787148452.640325:0:2394:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:1.0:1787148452.640327:0:2394:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c6f026440. 00000100:00000200:1.0:1787148452.640329:0:2394:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1873948276732416, offset 0 00000400:00000200:1.0:1787148452.640333:0:2394:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148452.641949:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.641962:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:0.0:1787148452.641967:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148452.641971:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148452.641975:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787148452.641978:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:0.0:1787148452.641982:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556899:556899:255:4294967295] 192.168.202.114@tcp LPNI seq info [556855:556855:7:4294967295] 00000400:00000200:0.0:1787148452.641987:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.641996:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787148452.642002:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148452.642011:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474ae800. 00000800:00000200:0.0:1787148452.642018:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:0.0:1787148452.642025:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:0.0:1787148452.642030:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474ae800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787148452.642036:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.642042:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: using a timeout of 24712 usecs (25 jiffies) 00000400:00000200:0.0:1787148452.642045:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:0.0:1787148452.642048:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306961436 00000100:00000001:0.0:1787148452.642052:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306965242 00000100:00000001:0.0:1787148452.642054:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306961436 00000100:00000001:0.0:1787148452.642056:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306965242 00000100:00000001:0.0:1787148452.642058:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148452.642061:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642065:0:2393:0:(client.c:1800:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1787148452.642072:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642100:0:2393:0:(client.c:1364:ptlrpc_import_delay_req()) Process entered 00000100:00000001:0.0:1787148452.642120:0:2393:0:(client.c:1423:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.642124:0:2393:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1787148452.642127:0:2393:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:0.0:1787148452.642130:0:2393:0:(sec.c:790:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:0.0:1787148452.642134:0:2393:0:(client.c:1906:ptlrpc_send_new_req()) Sending RPC req@ffff8c5c6dfeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732544:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148452.642139:0:2393:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1787148452.642144:0:2393:0:(sec.c:1086:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:0.0:1787148452.642147:0:2393:0:(sec.c:1123:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.642149:0:2393:0:(sec.c:1870:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:0.0:1787148452.642155:0:2393:0:(sec_null.c:178:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff8c5c4a665400. 02000000:00000001:0.0:1787148452.642157:0:2393:0:(sec.c:1880:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:0.0:1787148452.642161:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000010:0.0:1787148452.642165:0:2393:0:(lib-me.c:74:LNetMEAttach()) slab-alloced 'me': 88 at ffff8c5c483f4790. 00000400:00000010:0.0:1787148452.642171:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec440. 00000100:00000200:0.0:1787148452.642177:0:2393:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1873948276732544, portal 4 00000100:00000001:0.0:1787148452.642191:0:2393:0:(client.c:3262:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1787148452.642193:0:2393:0:(client.c:3264:ptlrpc_request_addref()) Process leaving (rc=18446616927343128832 : -127146366422784 : ffff8c5c6dfeb100) 00000100:00000040:0.0:1787148452.642201:0:2393:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642209:0:2393:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:0.0:1787148452.642212:0:2393:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.202.114@tcp 00000400:00000010:0.0:1787148452.642215:0:2393:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff8c5c47fec3b8. 00000100:00000200:0.0:1787148452.642218:0:2393:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1873948276732544, offset 0 00000400:00000200:0.0:1787148452.642223:0:2393:0:(lib-move.c:5157:LNetPut()) LNetPut -> 12345-192.168.202.114@tcp 00000400:00000200:0.0:1787148452.642229:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.642232:0:2393:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:0.0:1787148452.642235:0:2393:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1787148452.642238:0:2393:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:0.0:1787148452.642240:0:2393:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1787148452.642242:0:2393:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:0.0:1787148452.642251:0:2393:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556900:556900:254:4294967295] 192.168.202.114@tcp LPNI seq info [556856:556856:6:4294967295] 00000400:00000200:0.0:1787148452.642255:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.642262:0:2393:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:0.0:1787148452.642267:0:2393:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:0.0:1787148452.642271:0:2393:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c474ae500. 00000800:00000200:0.0:1787148452.642274:0:2393:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:0.0:1787148452.642279:0:2393:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:0.0:1787148452.642282:0:2393:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c474ae500 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:0.0:1787148452.642285:0:2393:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.642288:0:2393:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: using a timeout of 120655 usecs (121 jiffies) 00000400:00000200:0.0:1787148452.642291:0:2393:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:0.0:1787148452.642293:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306961533 00000100:00000001:0.0:1787148452.642296:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306965242 00000100:00000001:0.0:1787148452.642297:0:2393:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306961533 00000100:00000001:0.0:1787148452.642299:0:2393:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306965242 00000100:00000001:0.0:1787148452.642301:0:2393:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148452.642303:0:2393:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642307:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642311:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.642319:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148452.642321:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:0.0:1787148452.642324:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.642326:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148452.642328:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642330:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787148452.642411:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148452.642414:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7600. 00000400:00000200:0.0:1787148452.642422:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.642427:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148452.642433:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026000 00000400:00000010:0.0:1787148452.642435:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026000. 00000100:00000001:0.0:1787148452.642445:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148452.642451:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642461:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.642464:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642471:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642473:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1787148452.642486:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148452.642489:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474ae800. 00000400:00000200:0.0:1787148452.642491:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.642495:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148452.642498:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec7f8 00000400:00000010:0.0:1787148452.642500:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec7f8. 00000100:00000001:0.0:1787148452.642513:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148452.642517:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642525:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.642528:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642534:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642536:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:0.0:1787148452.642711:0:2385:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148452.642714:0:2385:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c474ae500. 00000400:00000200:0.0:1787148452.642717:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.642721:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148452.642724:0:2385:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec3b8 00000400:00000010:0.0:1787148452.642726:0:2385:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec3b8. 00000100:00000001:0.0:1787148452.642729:0:2385:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148452.642732:0:2385:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642740:0:2385:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.642760:0:2385:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.642767:0:2385:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.642769:0:2385:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1787148452.646695:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646716:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1787148452.646722:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646724:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.646735:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646743:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:0.0:1787148452.646765:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646770:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646775:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646779:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 25 MB=0x6a858955cb100 00000400:00000200:0.0:1787148452.646784:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646791:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e51 [1] + 224 00000800:00000001:0.0:1787148452.646798:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646802:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.646805:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.646809:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:0.0:1787148452.646826:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.646838:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787148452.646848:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.646857:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1787148452.646865:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646869:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1787148452.646871:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646873:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.646876:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646879:0:2386:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:0.0:1787148452.646883:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646905:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646908:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646911:0:2386:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 4 MB=0x6a858955cb280 00000400:00000200:0.0:1787148452.646915:0:2386:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.646920:0:2386:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e81 [1] + 224 00000800:00000001:0.0:1787148452.646924:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646930:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.646932:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.646935:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:0.0:1787148452.646938:0:2386:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.646943:0:2386:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787148452.646951:0:2386:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.646969:0:2386:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1787148452.646973:0:2386:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.646976:0:2386:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.646992:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.646996:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148452.647003:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148452.647005:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148452.647008:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f4790. 00000400:00000200:0.0:1787148452.647012:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec440 00000400:00000010:0.0:1787148452.647015:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec440. 00000100:00000001:0.0:1787148452.647018:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.647022:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148452.647031:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647037:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148452.647039:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647041:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148452.647049:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148452.647052:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148452.647054:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.647058:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148452.647068:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5e0b1c60 time=84 v=36 (36 5 5 5) 00000100:00001000:0.0:1787148452.647072:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5e0b1c18 time=60 v=5 (1 1 1 1) 00000100:00000001:0.0:1787148452.647074:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148452.647076:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148452.647079:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148452.647082:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647086:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148452.647088:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-OST0000-osc-ffff8c5c68fb1800: skip recheck: last_committed 42949695890 00000100:00000001:0.0:1787148452.647090:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148452.647092:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.647097:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647146:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148452.647150:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148452.647152:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148452.647156:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148452.647165:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfeb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732544:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148452.647169:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647172:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.647176:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfeb100 x1873948276732544/t0(0) o400->lustre-OST0000-osc-ffff8c5c68fb1800@192.168.202.114@tcp:28/4 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647182:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148452.647185:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148452.647189:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4a665400. 02000000:00000001:0.0:1787148452.647193:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148452.647195:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148452.647197:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5e0b1800 refcount=3 obd=lustre-OST0000-osc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148452.647199:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148452.647207:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474ae000. 02000000:00000001:0.0:1787148452.647211:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148452.647213:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148452.647215:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfeb100. 00000100:00000001:0.0:1787148452.647219:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148452.647220:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647223:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647225:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.647226:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1787148452.647228:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647230:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1787148452.647391:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.647397:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1787148452.647399:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.647401:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.647405:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.647409:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:0.0:1787148452.647414:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.647417:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.647420:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.647423:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cb180 00000400:00000200:0.0:1787148452.647427:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.647431:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e61 [1] + 224 00000800:00000001:0.0:1787148452.647435:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.647440:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.647442:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.647445:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:0.0:1787148452.647449:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.647454:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787148452.647463:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647475:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1787148452.647479:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.647488:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647495:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.647497:0:2393:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1787148452.647500:0:2393:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1787148452.647501:0:2393:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1787148452.647503:0:2393:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c483f40b0. 00000400:00000200:0.0:1787148452.647506:0:2393:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c47fec6e8 00000400:00000010:0.0:1787148452.647507:0:2393:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c47fec6e8. 00000100:00000001:0.0:1787148452.647510:0:2393:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.647514:0:2393:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1787148452.647522:0:2393:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647528:0:2393:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1787148452.647530:0:2393:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647531:0:2393:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:0.0:1787148452.647533:0:2393:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1787148452.647535:0:2393:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1787148452.647536:0:2393:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1787148452.647538:0:2393:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1787148452.647543:0:2393:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776a460 time=84 v=36 (36 36 36 36) 00000100:00001000:0.0:1787148452.647547:0:2393:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776a418 time=136 v=5 (1 1 0 0) 00000100:00000001:0.0:1787148452.647549:0:2393:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1787148452.647550:0:2393:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1787148452.647552:0:2393:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1787148452.647554:0:2393:0:(ldlm_request.c:1683:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647556:0:2393:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1787148452.647558:0:2393:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff8c5c68fb1800: skip recheck: last_committed 30064771460 00000100:00000001:0.0:1787148452.647560:0:2393:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1787148452.647561:0:2393:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1787148452.647643:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647651:0:2393:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1787148452.647653:0:2393:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1787148452.647654:0:2393:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1787148452.647664:0:2393:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1787148452.647671:0:2393:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2393:1873948276732288:192.168.202.114@tcp:400:kworker.0 00000100:00000001:0.0:1787148452.647677:0:2393:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647678:0:2393:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.647682:0:2393:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfea300 x1873948276732288/t0(0) o400->lustre-MDT0000-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.647688:0:2393:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1787148452.647689:0:2393:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1787148452.647691:0:2393:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c4a665e00. 02000000:00000001:0.0:1787148452.647694:0:2393:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1787148452.647695:0:2393:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1787148452.647697:0:2393:0:(genops.c:1244:class_import_put()) import ffff8c5c5776a000 refcount=5 obd=lustre-MDT0000-mdc-ffff8c5c68fb1800 00000020:00000001:0.0:1787148452.647698:0:2393:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1787148452.647700:0:2393:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474aee00. 02000000:00000001:0.0:1787148452.647702:0:2393:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1787148452.647704:0:2393:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1787148452.647705:0:2393:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfea300. 00000100:00000001:0.0:1787148452.647708:0:2393:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1787148452.647709:0:2393:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647711:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1787148452.647718:0:2393:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1787148452.647721:0:2393:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.647723:0:2393:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1787148452.647725:0:2393:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1787148452.648037:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148452.648045:0:2394:0:(lib-move.c:2813:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.202.114@tcp local destination 00000400:00000200:1.0:1787148452.648056:0:2394:0:(lib-move.c:2444:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1787148452.648062:0:2394:0:(lib-move.c:1625:lnet_get_best_ni()) selected best_ni 192.168.202.14@tcp 00000400:00000200:1.0:1787148452.648066:0:2394:0:(lib-move.c:1178:lnet_select_peer_ni()) 192.168.202.14@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1787148452.648069:0:2394:0:(lib-move.c:1247:lnet_select_peer_ni()) sd_best_lpni = 192.168.202.114@tcp 00000400:00000200:1.0:1787148452.648072:0:2394:0:(lib-move.c:1745:lnet_handle_send()) 192.168.202.14@tcp NI seq info: [556901:556901:256:4294967295] 192.168.202.114@tcp LPNI seq info [556857:556857:8:4294967295] 00000400:00000200:1.0:1787148452.648076:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:1.0:1787148452.648084:0:2394:0:(lib-move.c:1861:lnet_handle_send()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp:) -> 192.168.202.114@tcp(192.168.202.114@tcp:192.168.202.114@tcp) : PUT try# 0 00000800:00000200:1.0:1787148452.648091:0:2394:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.202.114@tcp 00000800:00000010:1.0:1787148452.648097:0:2394:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff8c5c68fa7a00. 00000800:00000200:1.0:1787148452.648118:0:2394:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff8c5c4508fcc0] -> 12345-192.168.202.114@tcp (5) 00000800:00000200:1.0:1787148452.648125:0:2394:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.202.114@tcp ip 192.168.202.114:988 00000800:00000200:1.0:1787148452.648129:0:2394:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff8c5c68fa7a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1787148452.648146:0:2394:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148452.648151:0:2394:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: using a timeout of 16988 usecs (17 jiffies) 00000400:00000200:1.0:1787148452.648155:0:2394:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000100:00000001:1.0:1787148452.648158:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 0 is 4306961435 00000100:00000001:1.0:1787148452.648161:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 0, keep current deadline 4306965242 00000100:00000001:1.0:1787148452.648163:0:2394:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS new deadline estimation for cpu 1 is 4306961435 00000100:00000001:1.0:1787148452.648165:0:2394:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff8c5c68fb1800: PM QoS request active for cpu 1, keep current deadline 4306965242 00000100:00000001:1.0:1787148452.648167:0:2394:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148452.648170:0:2394:0:(client.c:1928:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:1.0:1787148452.648174:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787148452.648175:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787148452.648177:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c482624d0. 00000400:00000200:1.0:1787148452.648180:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f0263b8 00000400:00000010:1.0:1787148452.649554:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f0263b8. 00000100:00000001:1.0:1787148452.649567:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148452.649577:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787148452.649597:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.649621:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787148452.649624:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.649627:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1787148452.649631:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787148452.649633:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787148452.649635:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.649638:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787148452.649644:0:2394:0:(import.c:2149:obd_at_measure()) add 5 to ffff8c5c5776b460 time=84 v=5 (5 5 5 5) 00000100:00001000:1.0:1787148452.649648:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776b418 time=136 v=5 (1 1 0 0) 00000100:00000001:1.0:1787148452.649650:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787148452.649652:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.649654:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00000001:1.0:1787148452.649656:0:2394:0:(ldlm_request.c:1657:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.649659:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148452.649664:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.649670:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148452.649672:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787148452.649674:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787148452.649677:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787148452.649685:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfeaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:adab7261-4d97-43ea-82e8-09637750fa21:2394:1873948276732160:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148452.649690:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.649693:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148452.649696:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfeaa00 x1873948276732160/t0(0) o400->MGC192.168.202.114@tcp@192.168.202.114@tcp:26/25 lens 224/224 e 0 to 0 dl 1787148468 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.649702:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787148452.649704:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787148452.649707:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c46ee7800. 02000000:00000001:1.0:1787148452.649711:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787148452.649713:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1787148452.649720:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776b000 refcount=3 obd=MGC192.168.202.114@tcp 00000020:00000001:1.0:1787148452.649722:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1787148452.649725:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474ae600. 02000000:00000001:1.0:1787148452.649728:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787148452.649730:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787148452.649732:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfeaa00. 00000100:00000001:1.0:1787148452.649735:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787148452.649736:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.649739:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.649746:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148452.649748:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=50 : 50 : 32) 00000100:00000001:1.0:1787148452.649751:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148452.649753:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1787148452.649756:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.649758:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1787148452.650578:0:2386:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:0.0:1787148452.650587:0:2386:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff8c5c68fa7a00. 00000400:00000200:0.0:1787148452.650602:0:2386:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.650608:0:2386:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000400:00000200:0.0:1787148452.650615:0:2386:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026440 00000400:00000010:0.0:1787148452.650617:0:2386:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026440. 00000100:00000001:0.0:1787148452.650622:0:2386:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:0.0:1787148452.650629:0:2386:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.650654:0:2386:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1787148452.650657:0:2386:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.650664:0:2386:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1787148452.650666:0:2386:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:0.0:1787148452.696389:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.696415:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1787148452.696420:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.696424:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.696436:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.696447:0:2385:0:(lib-move.c:4652:lnet_parse()) TRACE: 192.168.202.14@tcp(192.168.202.14@tcp) <- 192.168.202.114@tcp : PUT - for me 00000400:00000200:0.0:1787148452.696473:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.696479:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.696485:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.696490:0:2385:0:(lib-ptl.c:582:lnet_ptl_match_md()) Request from 12345-192.168.202.114@tcp of length 224 into portal 10 MB=0x6a858955cb200 00000400:00000200:0.0:1787148452.696495:0:2385:0:(api-ni.c:1613:lnet_nid4_cpt_hash()) Match nid 192.168.202.114@tcp to cpt 0 00000400:00000200:0.0:1787148452.696501:0:2385:0:(lib-ptl.c:208:lnet_try_match_md()) Incoming put index a from 12345-192.168.202.114@tcp of length 224/224 into md 0x561e71 [1] + 224 00000800:00000001:0.0:1787148452.696587:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.696605:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1787148452.696610:0:2385:0:(lib-msg.c:1121:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1787148452.696616:0:2385:0:(lib-msg.c:895:lnet_health_check()) health check: 192.168.202.14@tcp->192.168.202.114@tcp: PUT: OK 00000100:00000001:0.0:1787148452.696625:0:2385:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1787148452.696638:0:2385:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1787148452.696653:0:2385:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1787148452.696679:0:2385:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1787148452.696692:0:2385:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1787148452.696696:0:2385:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.696779:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148452.696786:0:2394:0:(client.c:1977:ptlrpc_check_set()) Process entered 00000400:00000001:1.0:1787148452.696796:0:2394:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:1.0:1787148452.696798:0:2394:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:1.0:1787148452.696803:0:2394:0:(lib-me.c:117:lnet_me_unlink()) slab-freed 'me': 88 at ffff8c5c48262f78. 00000400:00000200:1.0:1787148452.696810:0:2394:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff8c5c6f026198 00000400:00000010:1.0:1787148452.696824:0:2394:0:(lib-lnet.h:282:lnet_md_free()) slab-freed 'md': 136 at ffff8c5c6f026198. 00000100:00000001:1.0:1787148452.696829:0:2394:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1787148452.696835:0:2394:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:1.0:1787148452.696844:0:2394:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.696851:0:2394:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:1.0:1787148452.696853:0:2394:0:(client.c:2988:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.696856:0:2394:0:(client.c:1580:after_reply()) Process entered 02000000:00000001:1.0:1787148452.696860:0:2394:0:(sec.c:1131:do_cli_unwrap_reply()) Process entered 00000100:00000001:1.0:1787148452.696863:0:2394:0:(pack_generic.c:606:__lustre_unpack_msg()) Process entered 00000100:00000001:1.0:1787148452.696866:0:2394:0:(pack_generic.c:624:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1787148452.696869:0:2394:0:(sec.c:1193:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:1.0:1787148452.696882:0:2394:0:(import.c:2149:obd_at_measure()) add 36 to ffff8c5c5776cc60 time=84 v=36 (36 36 36 36) 00000100:00001000:1.0:1787148452.696887:0:2394:0:(import.c:2149:obd_at_measure()) add 1 to ffff8c5c5776cc18 time=111 v=5 (1 1 0 0) 00000100:00000001:1.0:1787148452.696890:0:2394:0:(client.c:1482:ptlrpc_check_status()) Process entered 00000100:00000001:1.0:1787148452.696912:0:2394:0:(client.c:1520:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.696916:0:2394:0:(ldlm_request.c:1653:ldlm_cli_update_pool()) Process entered 00010000:00080000:1.0:1787148452.696922:0:2394:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:1.0:1787148452.696930:0:2394:0:(ldlm_request.c:1672:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.696935:0:2394:0:(client.c:3096:ptlrpc_free_committed()) Process entered 00000100:00000040:1.0:1787148452.696938:0:2394:0:(client.c:3108:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff8c5c68fb1800: skip recheck: last_committed 21474836540 00000100:00000001:1.0:1787148452.696940:0:2394:0:(client.c:3109:ptlrpc_free_committed()) Process leaving 00000100:00000001:1.0:1787148452.696952:0:2394:0:(client.c:1784:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1787148452.696958:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.696965:0:2394:0:(client.c:2341:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1787148452.696970:0:2394:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:1.0:1787148452.696972:0:2394:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:1.0:1787148452.696976:0:2394:0:(lustre_net.h:2406:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:1.0:1787148452.696985:0:2394:0:(client.c:2401:ptlrpc_check_set()) Completed RPC req@ffff8c5c6dfe9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:7e31181a-25e1-4c2b-a23d-b0f7747006f1:2394:1873948276732416:192.168.202.114@tcp:400:kworker.0 00000100:00000001:1.0:1787148452.696990:0:2394:0:(client.c:2452:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.696994:0:2394:0:(client.c:2854:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1787148452.696998:0:2394:0:(client.c:2862:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff8c5c6dfe9180 x1873948276732416/t0(0) o400->lustre-MDT0001-mdc-ffff8c5c68fb1800@192.168.202.114@tcp:12/10 lens 224/224 e 0 to 0 dl 1787148507 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1787148452.697005:0:2394:0:(client.c:2780:__ptlrpc_free_req()) Process entered 02000000:00000001:1.0:1787148452.697008:0:2394:0:(sec.c:1895:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:1.0:1787148452.697013:0:2394:0:(sec_null.c:192:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff8c5c46ee6600. 02000000:00000001:1.0:1787148452.697018:0:2394:0:(sec.c:1911:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:1.0:1787148452.697021:0:2394:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:1.0:1787148452.697023:0:2394:0:(genops.c:1244:class_import_put()) import ffff8c5c5776c800 refcount=3 obd=lustre-MDT0001-mdc-ffff8c5c68fb1800 00000020:00000001:1.0:1787148452.697025:0:2394:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:1.0:1787148452.697029:0:2394:0:(sec_null.c:162:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff8c5c474aed00. 02000000:00000001:1.0:1787148452.697033:0:2394:0:(sec.c:479:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:1.0:1787148452.697035:0:2394:0:(sec.c:496:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:1.0:1787148452.697043:0:2394:0:(client.c:599:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff8c5c6dfe9180. 00000100:00000001:1.0:1787148452.697047:0:2394:0:(client.c:2836:__ptlrpc_free_req()) Process leaving 00000100:00000001:1.0:1787148452.697049:0:2394:0:(client.c:2895:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.697051:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1787148452.697060:0:2394:0:(client.c:2626:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1787148452.697062:0:2394:0:(client.c:2654:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1787148452.697064:0:2394:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1787148452.697067:0:2394:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.830631:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148452.830640:0:256195:0:(ldlm_request.c:2324:ldlm_cancel_lru()) Process entered 00010000:00000001:1.0:1787148452.830643:0:256195:0:(ldlm_request.c:2112:ldlm_prepare_lru_list()) Process entered 00010000:00000001:1.0:1787148452.830646:0:256195:0:(ldlm_request.c:2284:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.830651:0:256195:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:1.0:1787148452.830653:0:256195:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.830655:0:256195:0:(ldlm_request.c:2334:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.830658:0:256195:0:(ldlm_pool.c:498:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148452.830662:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148452.830664:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148452.831206:0:244719:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148452.831212:0:244719:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148452.831215:0:244719:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:1.0:1787148453.855709:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148453.855717:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148453.855723:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148453.855725:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1787148453.859860:0:244722:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1787148453.859867:0:244722:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1787148453.859872:0:244722:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1787148454.813525:0:277811:0:(llite_lib.c:4245:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148454.880584:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148454.880592:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148454.880599:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148454.880600:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1787148454.880799:0:256812:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1787148454.880803:0:256812:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1787148454.880807:0:256812:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000001:2.0:1787148455.416238:0:277847:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:2.0:1787148455.416245:0:277847:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_812a: @@@@@@ FAIL: client: import is not in CONNECTING state after 50 00000001:00000001:2.0:1787148455.436549:0:277847:0:(debug.c:656:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:1.0:1787148455.902179:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148455.902186:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148455.902192:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148455.902193:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1787148455.903368:0:235348:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1787148455.903374:0:235348:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1787148455.903378:0:235348:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1787148456.484406:0:256195:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342110 (cpu 1) has reached its deadline 4306965242, at 4306965253 00000100:00000010:1.0:1787148456.484424:0:256195:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e783840. 00000100:00000040:0.0:1787148456.484653:0:231971:0:(connection.c:75:cpu_latency_work()) work item of ffff8c5c48342000 (cpu 0) has reached its deadline 4306965242, at 4306965253 00000100:00000010:0.0:1787148456.484660:0:231971:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff8c5c5e7836c0. 00010000:00000001:1.0:1787148456.929328:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148456.929336:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:1.0:1787148456.929341:0:256195:0:(ldlm_pool.c:459:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:1.0:1787148456.929343:0:256195:0:(ldlm_pool.c:463:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1787148456.929367:0:235349:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1787148456.929370:0:235349:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1787148456.929388:0:235349:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1787148457.144631:0:277947:0:(llite_lib.c:4245:ll_show_options()) Process leaving (rc=0 : 0 : 0) Debug log: 2689 lines, 2689 kept, 0 dropped, 0 bad.