00000100:00000040:1.0F:1779879923.579832:0:275253:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355910 (cpu 0) has reached its deadline 4304993990, at 4304994048 00000100:00000010:1.0:1779879923.579854:0:275253:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f353bd1e0. 00000100:00000040:0.0F:1779879923.579993:0:287881:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355800 (cpu 0) has reached its deadline 4304993990, at 4304994048 00000100:00000010:0.0:1779879923.580006:0:287881:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f353bdba0. 00010000:00000001:0.0:1779879924.283791:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879924.283797:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879924.283803:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879924.283804:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0F:1779879924.283943:0:251675:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879924.283962:0:251675:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879924.283965:0:251675:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1779879924.860539:0:236939:0:(pinger.c:231:ptlrpc_pinger_process_import()) 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879924.860546:0:236939:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879924.860553:0:236939:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b5500. 00000020:00000040:1.0:1779879924.860559:0:236939:0:(genops.c:1231:class_import_get()) import ffff989f02475000 refcount=4 obd=MGC192.168.203.137@tcp 00000100:00000001:1.0:1779879924.860564:0:236939:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879924.860567:0:236939:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879924.860572:0:236939:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879924.860586:0:236939:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8800. 00000100:00000001:1.0:1779879924.860592:0:236939:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860603:0:236939:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879924.860614:0:236939:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879924.860618:0:236939:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860623:0:236939:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000d640f3e3] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860645:0:236939:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860647:0:236939:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879924.860651:0:236939:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879924.860653:0:236939:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b5c00. 00000020:00000040:1.0:1779879924.860655:0:236939:0:(genops.c:1231:class_import_get()) import ffff989f08d17800 refcount=9 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879924.860658:0:236939:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879924.860659:0:236939:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879924.860661:0:236939:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879924.860663:0:236939:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8900. 00000100:00000001:1.0:1779879924.860679:0:236939:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860684:0:236939:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879924.860691:0:236939:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879924.860693:0:236939:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860696:0:236939:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000aa7eded0] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860711:0:236939:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860713:0:236939:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879924.860716:0:236939:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879924.860718:0:236939:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b7480. 00000020:00000040:1.0:1779879924.860721:0:236939:0:(genops.c:1231:class_import_get()) import ffff989f08d17000 refcount=4 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879924.860735:0:236939:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879924.860736:0:236939:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879924.860738:0:236939:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879924.860742:0:236939:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8d00. 00000100:00000001:1.0:1779879924.860744:0:236939:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860748:0:236939:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879924.860763:0:236939:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879924.860764:0:236939:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860768:0:236939:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000c1502946] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860775:0:236939:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860777:0:236939:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879924.860780:0:236939:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879924.860782:0:236939:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b5880. 00000020:00000040:1.0:1779879924.860784:0:236939:0:(genops.c:1231:class_import_get()) import ffff989f08d11800 refcount=4 obd=lustre-OST0000-osc-ffff989f2f61e800 00000100:00000001:1.0:1779879924.860786:0:236939:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879924.860787:0:236939:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879924.860788:0:236939:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879924.860791:0:236939:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8a00. 00000100:00000001:1.0:1779879924.860793:0:236939:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860796:0:236939:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879924.860803:0:236939:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879924.860810:0:236939:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860814:0:236939:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ff7e3078] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860821:0:236939:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.860823:0:236939:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1779879924.860826:0:236939:0:(pinger.c:249:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1779879924.860831:0:236939:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (10333) 00000100:00000001:1.0:1779879924.860841:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879924.860844:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879924.860846:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879924.860850:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860857:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879924.860859:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.860862:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879924.860865:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879924.860867:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879924.860881:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2398:1866328745079424:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879924.860885:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879924.860890:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879924.860893:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.860895:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879924.860899:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f04f6ba00. 02000000:00000001:1.0:1779879924.860901:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879924.860909:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879924.860913:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07ebab58. 00000400:00000010:1.0:1779879924.860920:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac5d8. 00000100:00000200:1.0:1779879924.860927:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745079424, portal 25 00000100:00000001:1.0:1779879924.860939:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879924.860941:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195626240 : -113665513925376 : ffff989f2f6b5500) 00000100:00000040:1.0:1779879924.860946:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.860964:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879924.860968:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879924.860971:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9acf68. 00000100:00000200:1.0:1779879924.860974:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1866328745079424, offset 0 00000400:00000200:1.0:1779879924.860980:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879924.860988:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.860995:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879924.861000:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879924.861004:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879924.861007:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879924.861010:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879924.861014:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547888:547888:256:4294967295] 192.168.203.137@tcp LPNI seq info [547827:547827:8:4294967295] 00000400:00000200:1.0:1779879924.861018:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.861027:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879924.861033:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879924.861038:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8400. 00000800:00000200:1.0:1779879924.861247:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879924.861256:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879924.861260:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8400 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879924.861272:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879924.861277:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879924.861279:0:2398:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 0 is 4304999109 00000100:00000010:1.0:1779879924.861284:0:2398:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334240. 00000100:00000001:1.0:1779879924.861290:0:2398:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1779879924.861296:0:2398:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 1 is 4304999109 00000100:00000010:1.0:1779879924.861298:0:2398:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334c00. 00000100:00000001:1.0:1779879924.861301:0:2398:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1779879924.861304:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879924.861307:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.861315:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879924.861319:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.861326:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879924.861327:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.861329:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879924.861331:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879924.861333:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879924.861335:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079680:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879924.861338:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879924.861341:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879924.861342:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.861344:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879924.861347:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f04f6ac00. 02000000:00000001:1.0:1779879924.861349:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879924.861352:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879924.861355:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba580. 00000400:00000010:1.0:1779879924.861358:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac198. 00000100:00000200:1.0:1779879924.861362:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745079680, portal 10 00000100:00000001:1.0:1779879924.861364:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879924.861365:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195634304 : -113665513917312 : ffff989f2f6b7480) 00000100:00000040:1.0:1779879924.861371:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.861377:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879924.861379:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879924.861382:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac7f8. 00000100:00000200:1.0:1779879924.861383:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745079680, offset 0 00000400:00000200:1.0:1779879924.861387:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879924.861392:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.861396:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879924.861398:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879924.861400:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879924.861402:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879924.861410:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879924.861414:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547889:547889:255:4294967295] 192.168.203.137@tcp LPNI seq info [547828:547828:7:4294967295] 00000400:00000200:1.0:1779879924.861418:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.861424:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879924.861429:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879924.861432:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8300. 00000800:00000200:1.0:1779879924.861436:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879924.861440:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879924.861442:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879924.861445:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.861449:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: using a timeout of 24457 usecs (25 jiffies) 00000400:00000200:1.0:1779879924.861452:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879924.861453:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4304995353 00000100:00000001:1.0:1779879924.861456:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.861458:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4304995353 00000100:00000001:1.0:1779879924.861460:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.861461:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879924.861463:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.861466:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.861468:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879924.861475:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879924.861477:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1779879924.861479:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879924.861480:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879924.861482:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879924.861484:0:2398:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:1.0:1779879924.861486:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879924.861488:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879924.861489:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879924.861490:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879924.861492:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879924.861492:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879924.861502:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.861508:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879924.861510:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.861512:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879924.861513:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879924.861515:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879924.861517:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079552:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879924.861521:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879924.861523:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879924.861525:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.861526:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879924.861529:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f04f6a400. 02000000:00000001:1.0:1779879924.861530:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879924.861534:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879924.861536:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba630. 00000400:00000010:1.0:1779879924.861539:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac110. 00000100:00000200:1.0:1779879924.861542:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745079552, portal 10 00000100:00000001:1.0:1779879924.861544:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879924.861545:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195628032 : -113665513923584 : ffff989f2f6b5c00) 00000100:00000040:1.0:1779879924.861551:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.861557:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879924.861560:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879924.861562:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9accc0. 00000100:00000200:1.0:1779879924.861564:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745079552, offset 0 00000400:00000200:1.0:1779879924.861568:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879924.861572:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.861576:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879924.861578:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879924.861581:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879924.861583:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879924.862202:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879924.862206:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547890:547890:254:4294967295] 192.168.203.137@tcp LPNI seq info [547829:547829:6:4294967295] 00000400:00000200:1.0:1779879924.862211:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.862218:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879924.862223:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879924.862228:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8b00. 00000800:00000200:1.0:1779879924.862233:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879924.862237:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879924.862240:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8b00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879924.862244:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.862248:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: using a timeout of 16937 usecs (17 jiffies) 00000400:00000200:1.0:1779879924.862251:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879924.862253:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4304995345 00000100:00000001:1.0:1779879924.862256:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.862258:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4304995345 00000100:00000001:1.0:1779879924.862259:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.862261:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879924.862264:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862267:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879924.862272:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862280:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879924.862281:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.862284:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879924.862286:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879924.862289:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879924.862292:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079808:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879924.862295:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879924.862306:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879924.862308:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879924.862310:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879924.862313:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f04f6bc00. 02000000:00000001:1.0:1779879924.862315:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879924.862318:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879924.862321:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07ebabb0. 00000400:00000010:1.0:1779879924.862326:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac3b8. 00000100:00000200:1.0:1779879924.862329:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745079808, portal 4 00000100:00000001:1.0:1779879924.862331:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879924.862332:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195627136 : -113665513924480 : ffff989f2f6b5880) 00000100:00000040:1.0:1779879924.862337:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862342:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879924.862344:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879924.862346:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac220. 00000100:00000200:1.0:1779879924.862348:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1866328745079808, offset 0 00000400:00000200:1.0:1779879924.862352:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879924.862357:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.862360:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879924.862363:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879924.862365:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879924.862367:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879924.862369:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879924.862371:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547891:547891:253:4294967295] 192.168.203.137@tcp LPNI seq info [547830:547830:5:4294967295] 00000400:00000200:1.0:1779879924.862375:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.862381:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879924.862385:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879924.862388:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8e00. 00000800:00000200:1.0:1779879924.862392:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879924.862396:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879924.862398:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8e00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879924.862407:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879924.862410:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: using a timeout of 83380 usecs (84 jiffies) 00000400:00000200:1.0:1779879924.862414:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879924.862416:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4304995412 00000100:00000001:1.0:1779879924.862418:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.862420:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4304995412 00000100:00000001:1.0:1779879924.862422:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4304999109 00000100:00000001:1.0:1779879924.862424:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879924.862426:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862428:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862431:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862448:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879924.862450:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879924.862489:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879924.862493:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8400. 00000400:00000200:1.0:1779879924.862498:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.862504:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879924.862509:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9acf68 00000400:00000010:1.0:1779879924.862511:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9acf68. 00000100:00000001:1.0:1779879924.862515:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879924.862519:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862529:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879924.862532:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862538:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862539:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879924.862551:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879924.862553:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8300. 00000400:00000200:1.0:1779879924.862556:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.862559:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879924.862561:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac7f8 00000400:00000010:1.0:1779879924.862568:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac7f8. 00000100:00000001:1.0:1779879924.862571:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879924.862576:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862583:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879924.862585:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862591:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862593:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879924.862603:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879924.862605:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8b00. 00000400:00000200:1.0:1779879924.862608:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.862611:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879924.862613:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9accc0 00000400:00000010:1.0:1779879924.862615:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9accc0. 00000100:00000001:1.0:1779879924.862617:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879924.862620:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862627:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879924.862630:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.862635:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.862636:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879924.862709:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879924.862712:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8e00. 00000400:00000200:1.0:1779879924.862715:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.862718:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879924.862721:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac220 00000400:00000010:1.0:1779879924.862734:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac220. 00000100:00000001:1.0:1779879924.862736:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879924.862740:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.863069:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879924.863072:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.863077:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879924.863079:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1779879924.873910:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.873928:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879924.873942:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.873946:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.873956:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.873964:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879924.873975:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.873981:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.873986:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.873990:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27380 00000400:00000200:1.0:1779879924.873995:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874001:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fa81 [1] + 224 00000800:00000001:1.0:1779879924.874009:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874014:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.874017:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.874021:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879924.874028:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879924.874037:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879924.874048:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.874068:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879924.874077:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874081:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879924.874083:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874096:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.874100:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874103:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879924.874108:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874110:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874113:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874116:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27300 00000400:00000200:1.0:1779879924.874119:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874123:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fa91 [1] + 224 00000800:00000001:1.0:1779879924.874127:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874131:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.874133:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.874136:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879924.874139:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879924.874144:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879924.874153:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.874161:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879924.874165:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874168:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879924.874169:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874171:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.874174:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874177:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879924.874181:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874184:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874186:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874189:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 4 MB=0x6a16a85e27400 00000400:00000200:1.0:1779879924.874192:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.874196:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53faa1 [1] + 224 00000800:00000001:1.0:1779879924.874199:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874212:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.874214:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.874217:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879924.874221:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879924.874226:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879924.874234:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.874241:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879924.874245:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.874248:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874372:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879924.874377:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1779879924.874383:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879924.874385:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879924.874389:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba630. 00000400:00000200:0.0:1779879924.874405:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac110 00000400:00000010:0.0:1779879924.874407:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac110. 00000100:00000001:0.0:1779879924.874411:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879924.874416:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879924.874425:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874430:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879924.874432:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874435:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879924.874438:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879924.874440:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879924.874442:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879924.874445:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879924.874454:0:2398:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17c60 time=128 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879924.874459:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17c18 time=37 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879924.874462:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879924.874464:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879924.874466:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879924.874470:0:2398:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874473:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879924.874475:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff989f2f61e800: skip recheck: last_committed 30064771623 00000100:00000001:0.0:1779879924.874476:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879924.874478:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879924.874482:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874496:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879924.874500:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879924.874502:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879924.874506:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879924.874512:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079552:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879924.874518:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879924.874519:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879924.874520:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07ebabb0. 00000400:00000200:0.0:1779879924.874523:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac3b8 00000400:00000010:0.0:1779879924.874524:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac3b8. 00000100:00000001:0.0:1779879924.874527:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879924.874529:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879924.874536:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874542:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879924.874543:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874545:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879924.874547:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879924.874548:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879924.874549:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879924.874551:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879924.874555:0:2398:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f08d11c60 time=128 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879924.874558:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d11c18 time=85 v=5 (1 1 0 0) 00000100:00000001:0.0:1779879924.874560:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879924.874561:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879924.874563:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879924.874565:0:2398:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874567:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879924.874568:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-OST0000-osc-ffff989f2f61e800: skip recheck: last_committed 68719497264 00000100:00000001:0.0:1779879924.874586:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879924.874588:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879924.874593:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874599:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879924.874601:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879924.874603:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879924.874606:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879924.874612:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079808:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879924.874617:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879924.874619:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879924.874620:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba580. 00000400:00000200:0.0:1779879924.874623:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac198 00000400:00000010:0.0:1779879924.874624:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac198. 00000100:00000001:0.0:1779879924.874626:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879924.874629:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879924.874636:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874642:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879924.874644:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874645:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879924.874647:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879924.874648:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879924.874649:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879924.874651:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879924.874656:0:2398:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17460 time=128 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879924.874658:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17418 time=15 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879924.874660:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879924.874661:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879924.874696:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1779879924.874701:0:2398:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1779879924.874707:0:2398:0:(ldlm_request.c:1664:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874709:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879924.874711:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff989f2f61e800: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1779879924.874713:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879924.874714:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879924.874718:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874724:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879924.874726:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879924.874727:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879924.874730:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879924.874737:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745079680:192.168.203.137@tcp:400:kworker.0 00000100:00000001:0.0:1779879924.874741:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874744:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879924.874747:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b5c00 x1866328745079552/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874753:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879924.874755:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879924.874758:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f04f6a400. 02000000:00000001:0.0:1779879924.874762:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879924.874765:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879924.874767:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d17800 refcount=8 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879924.874769:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879924.874771:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8900. 02000000:00000001:0.0:1779879924.874774:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879924.874776:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879924.874778:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b5c00. 00000100:00000001:0.0:1779879924.874787:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879924.874789:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874790:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879924.874794:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b5880 x1866328745079808/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874799:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879924.874801:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879924.874803:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f04f6bc00. 02000000:00000001:0.0:1779879924.874805:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879924.874806:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879924.874807:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d11800 refcount=3 obd=lustre-OST0000-osc-ffff989f2f61e800 00000020:00000001:0.0:1779879924.874809:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879924.874810:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8a00. 02000000:00000001:0.0:1779879924.874812:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879924.874814:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879924.874815:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b5880. 00000100:00000001:0.0:1779879924.874817:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879924.874818:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874819:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879924.874822:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b7480 x1866328745079680/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880009 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.874829:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879924.874830:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879924.874831:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f04f6ac00. 02000000:00000001:0.0:1779879924.874833:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879924.874834:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879924.874835:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d17000 refcount=3 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879924.874837:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879924.874838:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8d00. 02000000:00000001:0.0:1779879924.874840:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879924.874841:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879924.874842:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b7480. 00000100:00000001:0.0:1779879924.874844:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879924.874845:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.874848:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874849:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879924.874856:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:0.0:1779879924.874858:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.874860:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:1.0:1779879924.879552:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.879583:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879924.879605:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.879609:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.879617:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.879624:0:2391:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879924.879633:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.879637:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.879642:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.879646:0:2391:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 25 MB=0x6a16a85e27280 00000400:00000200:1.0:1779879924.879651:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879924.879657:0:2391:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fa71 [1] + 224 00000800:00000001:1.0:1779879924.879664:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.879711:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879924.879721:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879924.879735:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879924.879743:0:2391:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879924.879751:0:2391:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879924.879762:0:2391:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879924.879795:0:2391:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879924.879812:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879924.879815:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.880141:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879924.880145:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1779879924.880151:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879924.880153:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879924.880156:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07ebab58. 00000400:00000200:0.0:1779879924.880167:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac5d8 00000400:00000010:0.0:1779879924.880169:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac5d8. 00000100:00000001:0.0:1779879924.880172:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879924.880177:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879924.880190:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.880196:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879924.880198:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.880201:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879924.880204:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879924.880207:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879924.880209:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879924.880212:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879924.880218:0:2398:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f02475460 time=128 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879924.880228:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f02475418 time=37 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879924.880231:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879924.880232:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879924.880235:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879924.880236:0:2398:0:(ldlm_request.c:1649:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.880239:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879924.880243:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.880249:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879924.880259:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879924.880261:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879924.880265:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879924.880295:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2398:1866328745079424:192.168.203.137@tcp:400:kworker.0 00000100:00000001:0.0:1779879924.880300:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.880302:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879924.880305:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b5500 x1866328745079424/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879940 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879924.880319:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879924.880322:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879924.880327:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f04f6ba00. 02000000:00000001:0.0:1779879924.880331:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879924.880333:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879924.880334:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f02475000 refcount=3 obd=MGC192.168.203.137@tcp 00000020:00000001:0.0:1779879924.880336:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879924.880339:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8800. 02000000:00000001:0.0:1779879924.880342:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879924.880351:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879924.880353:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b5500. 00000100:00000001:0.0:1779879924.880357:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879924.880359:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.880361:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879924.880369:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1779879924.880371:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879924.880372:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879924.880375:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879925.309083:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879925.309089:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879925.309095:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879925.309097:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0F:1779879925.309236:0:245969:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1779879925.309245:0:245969:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1779879925.309249:0:245969:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879926.332022:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879926.332026:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879926.332031:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879926.332035:0:287881:0:(ldlm_request.c:2313:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1779879926.332037:0:287881:0:(ldlm_request.c:2101:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1779879926.332043:0:287881:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1779879926.332045:0:287881:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1779879926.332065:0:287881:0:(ldlm_request.c:2273:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879926.332068:0:287881:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1779879926.332069:0:287881:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879926.332071:0:287881:0:(ldlm_request.c:2323:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879926.332073:0:287881:0:(ldlm_pool.c:497:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879926.336799:0:239871:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879926.336804:0:239871:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879926.336807:0:239871:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879927.355800:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879927.355806:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879927.355812:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879927.355814:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879927.355948:0:251676:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879927.355952:0:251676:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879927.355954:0:251676:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879928.379822:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879928.379827:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879928.380120:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879928.380122:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879928.380251:0:239870:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879928.380254:0:239870:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879928.380256:0:239870:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1779879928.700755:0:287881:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355800 (cpu 0) has reached its deadline 4304999109, at 4304999169 00000100:00000010:0.0:1779879928.700770:0:287881:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f1e334240. 00000100:00000040:1.0:1779879928.701131:0:275253:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355910 (cpu 0) has reached its deadline 4304999109, at 4304999169 00000100:00000010:1.0:1779879928.701139:0:275253:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f1e334c00. 00010000:00000001:0.0:1779879929.410982:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879929.410988:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879929.410994:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879929.410995:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879929.411118:0:251675:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879929.411122:0:251675:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879929.411125:0:251675:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:3.0:1779879929.541016:0:288128:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981234:0:255124:0:(pinger.c:231:ptlrpc_pinger_process_import()) 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879929.981245:0:255124:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879929.981251:0:255124:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b6a00. 00000020:00000040:1.0:1779879929.981256:0:255124:0:(genops.c:1231:class_import_get()) import ffff989f02475000 refcount=4 obd=MGC192.168.203.137@tcp 00000100:00000001:1.0:1779879929.981260:0:255124:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879929.981264:0:255124:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879929.981269:0:255124:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879929.981276:0:255124:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8500. 00000100:00000001:1.0:1779879929.981283:0:255124:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981295:0:255124:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879929.981305:0:255124:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879929.981309:0:255124:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981315:0:255124:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005a66ea34] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981335:0:255124:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981338:0:255124:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879929.981341:0:255124:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879929.981344:0:255124:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b7b80. 00000020:00000040:1.0:1779879929.981347:0:255124:0:(genops.c:1231:class_import_get()) import ffff989f08d17800 refcount=9 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879929.981349:0:255124:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879929.981351:0:255124:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879929.981352:0:255124:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879929.981356:0:255124:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8400. 00000100:00000001:1.0:1779879929.981358:0:255124:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981370:0:255124:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879929.981377:0:255124:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879929.981379:0:255124:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981383:0:255124:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000621cfd5b] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981391:0:255124:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981393:0:255124:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879929.981397:0:255124:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879929.981399:0:255124:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b7100. 00000020:00000040:1.0:1779879929.981402:0:255124:0:(genops.c:1231:class_import_get()) import ffff989f08d17000 refcount=4 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879929.981403:0:255124:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879929.981405:0:255124:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879929.981406:0:255124:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879929.981409:0:255124:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8300. 00000100:00000001:1.0:1779879929.981412:0:255124:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981415:0:255124:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879929.981422:0:255124:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879929.981423:0:255124:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981427:0:255124:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000013cb743e] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981434:0:255124:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981436:0:255124:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879929.981439:0:255124:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879929.981441:0:255124:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b6d80. 00000020:00000040:1.0:1779879929.981443:0:255124:0:(genops.c:1231:class_import_get()) import ffff989f08d11800 refcount=4 obd=lustre-OST0000-osc-ffff989f2f61e800 00000100:00000001:1.0:1779879929.981445:0:255124:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879929.981446:0:255124:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879929.981448:0:255124:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879929.981455:0:255124:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8800. 00000100:00000001:1.0:1779879929.981459:0:255124:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981463:0:255124:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879929.981469:0:255124:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879929.981471:0:255124:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981474:0:255124:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000dfa6f1f7] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981481:0:255124:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.981483:0:255124:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1779879929.981486:0:255124:0:(pinger.c:249:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000040:1.0:1779879929.981491:0:255124:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (10338) 00000100:00000001:1.0:1779879929.981502:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879929.981505:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879929.981508:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879929.981512:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981520:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879929.981522:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.981525:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879929.981528:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879929.981530:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879929.981533:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2398:1866328745079936:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879929.981537:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879929.981542:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879929.981544:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.981547:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879929.981551:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f62400. 02000000:00000001:1.0:1779879929.981553:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879929.981559:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879929.981563:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba210. 00000400:00000010:1.0:1779879929.981574:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac220. 00000100:00000200:1.0:1779879929.981580:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745079936, portal 25 00000100:00000001:1.0:1779879929.981582:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879929.981584:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195631616 : -113665513920000 : ffff989f2f6b6a00) 00000100:00000040:1.0:1779879929.981590:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.981597:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879929.982885:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879929.982892:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9accc0. 00000100:00000200:1.0:1779879929.982898:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1866328745079936, offset 0 00000400:00000200:1.0:1779879929.982906:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879929.982916:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.982923:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879929.982929:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879929.982933:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879929.982936:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879929.982938:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879929.982943:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547892:547892:256:4294967295] 192.168.203.137@tcp LPNI seq info [547831:547831:8:4294967295] 00000400:00000200:1.0:1779879929.982948:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.982958:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879929.982965:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879929.982970:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8d00. 00000800:00000200:1.0:1779879929.982978:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879929.982986:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879929.982990:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8d00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879929.983004:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879929.983008:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879929.983011:0:2398:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 0 is 4305004232 00000100:00000010:1.0:1779879929.983016:0:2398:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334960. 00000100:00000001:1.0:1779879929.983022:0:2398:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1779879929.983028:0:2398:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 1 is 4305004232 00000100:00000010:1.0:1779879929.983035:0:2398:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334840. 00000100:00000001:1.0:1779879929.983038:0:2398:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1779879929.983040:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879929.983043:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.983047:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879929.983052:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.983058:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879929.983060:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.983772:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879929.983777:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879929.983780:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879929.983786:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080192:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879929.983791:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879929.983797:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879929.983800:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.983803:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879929.983808:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f62a00. 02000000:00000001:1.0:1779879929.983810:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879929.983815:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879929.983819:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07ebab00. 00000400:00000010:1.0:1779879929.983825:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac7f8. 00000100:00000200:1.0:1779879929.983839:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080192, portal 10 00000100:00000001:1.0:1779879929.983842:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879929.983844:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195633408 : -113665513918208 : ffff989f2f6b7100) 00000100:00000040:1.0:1779879929.983851:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.983858:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879929.983861:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879929.983864:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9acf68. 00000100:00000200:1.0:1779879929.983867:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745080192, offset 0 00000400:00000200:1.0:1779879929.983872:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879929.983888:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.983893:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879929.983897:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879929.983900:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879929.983903:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879929.983906:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879929.983909:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547893:547893:255:4294967295] 192.168.203.137@tcp LPNI seq info [547832:547832:7:4294967295] 00000400:00000200:1.0:1779879929.983913:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.983921:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879929.983926:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879929.983930:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8a00. 00000800:00000200:1.0:1779879929.983934:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879929.983940:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879929.983943:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8a00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879929.983946:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.983950:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: using a timeout of 24457 usecs (25 jiffies) 00000400:00000200:1.0:1779879929.983954:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879929.983956:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305000477 00000100:00000001:1.0:1779879929.983959:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.983961:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305000477 00000100:00000001:1.0:1779879929.983964:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.983966:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879929.983968:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.983972:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.983975:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879929.983982:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879929.983984:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1779879929.983987:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879929.983988:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879929.983990:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879929.983994:0:2398:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00000001:1.0:1779879929.984001:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879929.984004:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879929.984005:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879929.984007:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879929.984008:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879929.984009:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879929.984014:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.984020:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879929.984022:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.984024:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879929.984026:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879929.984028:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879929.984030:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080064:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879929.984033:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879929.984036:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879929.984037:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.984039:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879929.984042:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f63800. 02000000:00000001:1.0:1779879929.984043:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879929.984046:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879929.984048:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba6e0. 00000400:00000010:1.0:1779879929.984051:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac880. 00000100:00000200:1.0:1779879929.984053:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080064, portal 10 00000100:00000001:1.0:1779879929.984055:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879929.984056:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195636096 : -113665513915520 : ffff989f2f6b7b80) 00000100:00000040:1.0:1779879929.984062:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.984069:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879929.984072:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879929.984075:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac5d8. 00000100:00000200:1.0:1779879929.984077:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745080064, offset 0 00000400:00000200:1.0:1779879929.984083:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879929.984088:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.984098:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879929.984101:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879929.984103:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879929.984107:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879929.984109:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879929.984112:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547894:547894:254:4294967295] 192.168.203.137@tcp LPNI seq info [547833:547833:6:4294967295] 00000400:00000200:1.0:1779879929.984117:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.984124:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879929.985215:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879929.985220:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8900. 00000800:00000200:1.0:1779879929.985227:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879929.985233:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879929.985236:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8900 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879929.985241:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.985246:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: using a timeout of 16937 usecs (17 jiffies) 00000400:00000200:1.0:1779879929.985250:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879929.985252:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305000470 00000100:00000001:1.0:1779879929.985255:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.985256:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305000470 00000100:00000001:1.0:1779879929.985258:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.985260:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879929.985263:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.985267:0:2398:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879929.985272:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.985279:0:2398:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879929.985281:0:2398:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.985284:0:2398:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879929.985287:0:2398:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879929.985294:0:2398:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879929.985298:0:2398:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080320:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879929.985301:0:2398:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879929.985305:0:2398:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879929.985308:0:2398:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879929.985310:0:2398:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879929.985314:0:2398:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f62e00. 02000000:00000001:1.0:1779879929.985316:0:2398:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879929.985320:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879929.985323:0:2398:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba688. 00000400:00000010:1.0:1779879929.985328:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac198. 00000100:00000200:1.0:1779879929.985333:0:2398:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080320, portal 4 00000100:00000001:1.0:1779879929.985335:0:2398:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879929.985336:0:2398:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195632512 : -113665513919104 : ffff989f2f6b6d80) 00000100:00000040:1.0:1779879929.985343:0:2398:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.985349:0:2398:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879929.985351:0:2398:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879929.985353:0:2398:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac3b8. 00000100:00000200:1.0:1779879929.985355:0:2398:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1866328745080320, offset 0 00000400:00000200:1.0:1779879929.985360:0:2398:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879929.985364:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.985369:0:2398:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879929.985372:0:2398:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879929.985374:0:2398:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879929.985376:0:2398:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879929.985379:0:2398:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879929.985381:0:2398:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547895:547895:253:4294967295] 192.168.203.137@tcp LPNI seq info [547834:547834:5:4294967295] 00000400:00000200:1.0:1779879929.985385:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879929.985392:0:2398:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879929.985396:0:2398:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879929.985405:0:2398:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f0a08e300. 00000800:00000200:1.0:1779879929.985410:0:2398:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879929.985414:0:2398:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879929.985417:0:2398:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f0a08e300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879929.985420:0:2398:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879929.985423:0:2398:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: using a timeout of 83335 usecs (84 jiffies) 00000400:00000200:1.0:1779879929.985426:0:2398:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879929.985428:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305000537 00000100:00000001:1.0:1779879929.985430:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.985432:0:2398:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305000537 00000100:00000001:1.0:1779879929.985434:0:2398:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305004232 00000100:00000001:1.0:1779879929.985436:0:2398:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879929.985438:0:2398:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.985441:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.985444:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.985462:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879929.985465:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879929.985503:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879929.985507:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8d00. 00000400:00000200:1.0:1779879929.985513:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879929.985518:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879929.985523:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9accc0 00000400:00000010:1.0:1779879929.985525:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9accc0. 00000100:00000001:1.0:1779879929.985529:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879929.985534:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.985545:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879929.985548:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.985555:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.985557:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879929.985574:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879929.985576:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8a00. 00000400:00000200:1.0:1779879929.985579:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879929.985583:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879929.985585:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9acf68 00000400:00000010:1.0:1779879929.985587:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9acf68. 00000100:00000001:1.0:1779879929.985589:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879929.985593:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986708:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879929.986718:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986726:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.986728:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879929.986748:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879929.986750:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8900. 00000400:00000200:1.0:1779879929.986754:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879929.986758:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879929.986761:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac5d8 00000400:00000010:1.0:1779879929.986764:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac5d8. 00000100:00000001:1.0:1779879929.986767:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879929.986771:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986778:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879929.986781:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986787:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.986788:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1779879929.986867:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879929.986870:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f0a08e300. 00000400:00000200:1.0:1779879929.986873:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879929.986876:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879929.986879:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac3b8 00000400:00000010:1.0:1779879929.986888:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac3b8. 00000100:00000001:1.0:1779879929.986891:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879929.986895:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986902:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879929.986905:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879929.986911:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879929.986913:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:1.0:1779879930.000994:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001012:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879930.001018:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001021:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001029:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001039:0:2391:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879930.001050:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001055:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001061:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001066:0:2391:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 25 MB=0x6a16a85e27480 00000400:00000200:1.0:1779879930.001073:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001080:0:2391:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fab1 [1] + 224 00000800:00000001:1.0:1779879930.001087:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001091:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001094:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879930.001098:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879930.001104:0:2391:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879930.001111:0:2391:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879930.001121:0:2391:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879930.001143:0:2391:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879930.001161:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001165:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879930.001167:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001169:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001172:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001175:0:2391:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879930.001179:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001182:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001185:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001187:0:2391:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27580 00000400:00000200:1.0:1779879930.001191:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001195:0:2391:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fac1 [1] + 224 00000800:00000001:1.0:1779879930.001199:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001201:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001203:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879930.001206:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879930.001209:0:2391:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879930.001214:0:2391:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879930.001221:0:2391:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879930.001229:0:2391:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879930.001234:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001237:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879930.001239:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001242:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001245:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001248:0:2391:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879930.001252:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001254:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001257:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001259:0:2391:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27500 00000400:00000200:1.0:1779879930.001262:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001271:0:2391:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fad1 [1] + 224 00000800:00000001:1.0:1779879930.001275:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001279:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001281:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879930.001284:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879930.001287:0:2391:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879930.001292:0:2391:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879930.001300:0:2391:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879930.001308:0:2391:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879930.001313:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001315:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:1.0:1779879930.001317:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001320:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001323:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001326:0:2391:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:1.0:1779879930.001330:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001332:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001335:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001337:0:2391:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 4 MB=0x6a16a85e27600 00000400:00000200:1.0:1779879930.001341:0:2391:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879930.001345:0:2391:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fae1 [1] + 224 00000800:00000001:1.0:1779879930.001350:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001357:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:1.0:1779879930.001359:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879930.001362:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:1.0:1779879930.001365:0:2391:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:1.0:1779879930.001369:0:2391:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:1.0:1779879930.001377:0:2391:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879930.001390:0:2391:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:1.0:1779879930.001396:0:2391:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:1.0:1779879930.001399:0:2391:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001448:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879930.001452:0:2398:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1779879930.001458:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879930.001460:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879930.001463:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba6e0. 00000400:00000200:0.0:1779879930.001468:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac880 00000400:00000010:0.0:1779879930.001471:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac880. 00000100:00000001:0.0:1779879930.001474:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879930.001478:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879930.001487:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001493:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879930.001495:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001497:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879930.001500:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879930.001503:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879930.001510:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879930.001514:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879930.001523:0:2398:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17c60 time=134 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879930.001528:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17c18 time=43 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879930.001530:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879930.001532:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.001535:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879930.001539:0:2398:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879930.001542:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879930.001545:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff989f2f61e800: skip recheck: last_committed 30064771623 00000100:00000001:0.0:1779879930.001548:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879930.001550:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879930.001555:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001562:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879930.001565:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879930.001567:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879930.001571:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879930.001579:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080064:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879930.001585:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879930.001586:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879930.001588:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba688. 00000400:00000200:0.0:1779879930.001591:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac198 00000400:00000010:0.0:1779879930.001592:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac198. 00000100:00000001:0.0:1779879930.001594:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879930.001597:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879930.001605:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001611:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879930.001618:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001620:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879930.001622:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879930.001623:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879930.001624:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879930.001626:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879930.001630:0:2398:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f08d11c60 time=134 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879930.001633:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d11c18 time=91 v=5 (1 1 0 0) 00000100:00000001:0.0:1779879930.001635:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879930.001637:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.001638:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879930.001641:0:2398:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879930.001643:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879930.001645:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-OST0000-osc-ffff989f2f61e800: skip recheck: last_committed 68719497264 00000100:00000001:0.0:1779879930.001647:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879930.001649:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879930.001653:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001659:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879930.001661:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879930.001662:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879930.001666:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879930.001716:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080320:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879930.001722:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879930.001723:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879930.001725:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba210. 00000400:00000200:0.0:1779879930.001728:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac220 00000400:00000010:0.0:1779879930.001729:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac220. 00000100:00000001:0.0:1779879930.001732:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879930.001735:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879930.001743:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001755:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879930.001757:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001759:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879930.001760:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879930.001762:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879930.001764:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879930.001766:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879930.001769:0:2398:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f02475460 time=134 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879930.001771:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f02475418 time=43 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879930.001773:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879930.001774:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.001776:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879930.001777:0:2398:0:(ldlm_request.c:1649:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879930.001779:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879930.001783:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001788:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879930.001789:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879930.001790:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879930.001792:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879930.001799:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2398:1866328745079936:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879930.001804:0:2398:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879930.001805:0:2398:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879930.001806:0:2398:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07ebab00. 00000400:00000200:0.0:1779879930.001808:0:2398:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac7f8 00000400:00000010:0.0:1779879930.001810:0:2398:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac7f8. 00000100:00000001:0.0:1779879930.001812:0:2398:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879930.001814:0:2398:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879930.001821:0:2398:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001843:0:2398:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879930.001845:0:2398:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001846:0:2398:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879930.001848:0:2398:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879930.001849:0:2398:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879930.001851:0:2398:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879930.001852:0:2398:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879930.001856:0:2398:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17460 time=134 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879930.001859:0:2398:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17418 time=21 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879930.001860:0:2398:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879930.001861:0:2398:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.001863:0:2398:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1779879930.001866:0:2398:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1779879930.001872:0:2398:0:(ldlm_request.c:1664:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879930.001875:0:2398:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879930.001877:0:2398:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff989f2f61e800: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1779879930.001878:0:2398:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879930.001880:0:2398:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879930.001883:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001890:0:2398:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879930.001891:0:2398:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879930.001892:0:2398:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879930.001896:0:2398:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879930.001902:0:2398:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2398:1866328745080192:192.168.203.137@tcp:400:kworker.0 00000100:00000001:0.0:1779879930.001906:0:2398:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001908:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879930.001911:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b7b80 x1866328745080064/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001925:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879930.001928:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879930.001931:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f63800. 02000000:00000001:0.0:1779879930.001935:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879930.001938:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879930.001939:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d17800 refcount=8 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879930.001941:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879930.001943:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8400. 02000000:00000001:0.0:1779879930.001947:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879930.001949:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879930.001951:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b7b80. 00000100:00000001:0.0:1779879930.001956:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879930.001957:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001959:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879930.001962:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b6d80 x1866328745080320/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.001969:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879930.001970:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879930.001972:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f62e00. 02000000:00000001:0.0:1779879930.001974:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879930.001975:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879930.001976:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d11800 refcount=3 obd=lustre-OST0000-osc-ffff989f2f61e800 00000020:00000001:0.0:1779879930.001978:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879930.001980:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8800. 02000000:00000001:0.0:1779879930.001981:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879930.001982:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879930.001984:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b6d80. 00000100:00000001:0.0:1779879930.001986:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879930.001988:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.001989:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879930.001993:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b6a00 x1866328745079936/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879945 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.002000:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879930.002001:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879930.002008:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f62400. 02000000:00000001:0.0:1779879930.002010:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879930.002011:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879930.002013:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f02475000 refcount=3 obd=MGC192.168.203.137@tcp 00000020:00000001:0.0:1779879930.002015:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879930.002016:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8500. 02000000:00000001:0.0:1779879930.002018:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879930.002019:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879930.002020:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b6a00. 00000100:00000001:0.0:1779879930.002022:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879930.002024:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.002025:0:2398:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879930.002028:0:2398:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b7100 x1866328745080192/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880014 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879930.002034:0:2398:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879930.002035:0:2398:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879930.002037:0:2398:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f62a00. 02000000:00000001:0.0:1779879930.002038:0:2398:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879930.002039:0:2398:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879930.002040:0:2398:0:(genops.c:1244:class_import_put()) import ffff989f08d17000 refcount=3 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879930.002042:0:2398:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879930.002043:0:2398:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8300. 02000000:00000001:0.0:1779879930.002045:0:2398:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879930.002046:0:2398:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879930.002048:0:2398:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b7100. 00000100:00000001:0.0:1779879930.002050:0:2398:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879930.002052:0:2398:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.002054:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879930.002060:0:2398:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1779879930.002061:0:2398:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879930.002063:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879930.002065:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.427861:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879930.427868:0:287881:0:(ldlm_request.c:2313:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1779879930.427870:0:287881:0:(ldlm_request.c:2101:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1779879930.427874:0:287881:0:(ldlm_request.c:2273:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.427879:0:287881:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1779879930.427895:0:287881:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.427897:0:287881:0:(ldlm_request.c:2323:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.427900:0:287881:0:(ldlm_pool.c:497:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879930.427904:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879930.427906:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1779879930.427990:0:245969:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1779879930.427994:0:245969:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1779879930.427996:0:245969:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879931.451884:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879931.451890:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879931.451896:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879931.451897:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879931.451969:0:239871:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879931.451972:0:239871:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879931.451975:0:239871:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879932.475794:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879932.475799:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879932.475805:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879932.475806:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1779879932.476481:0:251676:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1779879932.476485:0:251676:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1779879932.476488:0:251676:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879933.499785:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879933.499791:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879933.499797:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879933.499798:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879933.499913:0:239870:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879933.499917:0:239870:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879933.499919:0:239870:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1779879933.819849:0:275253:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355910 (cpu 0) has reached its deadline 4305004232, at 4305004288 00000100:00000010:1.0:1779879933.819863:0:275253:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f1e334840. 00000100:00000040:0.0:1779879933.819920:0:287881:0:(connection.c:75:cpu_latency_work()) work item of ffff989f06355800 (cpu 0) has reached its deadline 4305004232, at 4305004288 00000100:00000010:0.0:1779879933.819926:0:287881:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at ffff989f1e334960. 00010000:00000001:0.0:1779879934.523942:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879934.523948:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879934.523954:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879934.523955:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1779879934.524198:0:251675:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1779879934.524201:0:251675:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1779879934.524204:0:251675:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:1.0:1779879935.099979:0:214361:0:(pinger.c:231:ptlrpc_pinger_process_import()) 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879935.099987:0:214361:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:1.0:1779879935.099994:0:214361:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b4e00. 00000020:00000040:1.0:1779879935.099999:0:214361:0:(genops.c:1231:class_import_get()) import ffff989f02475000 refcount=4 obd=MGC192.168.203.137@tcp 00000100:00000001:1.0:1779879935.100003:0:214361:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879935.100006:0:214361:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879935.100011:0:214361:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879935.100017:0:214361:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f0a08eb00. 00000100:00000001:1.0:1779879935.100023:0:214361:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100034:0:214361:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 93850f6c-6c43-4b8a-8d90-a4353df8a1d5->MGS req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879935.100044:0:214361:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879935.100048:0:214361:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100054:0:214361:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000005b827870] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100074:0:214361:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100076:0:214361:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879935.100085:0:214361:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1779879935.100092:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00100000:1.0:1779879935.100096:0:2399:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1779879935.100098:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.100105:0:2399:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879935.100107:0:2399:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.100109:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.100111:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.100114:0:2399:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879935.100119:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100125:0:2399:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879935.100127:0:2399:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100131:0:2399:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879935.100133:0:2399:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879935.100136:0:2399:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879935.100139:0:2399:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2399:1866328745080448:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879935.100142:0:2399:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879935.100147:0:2399:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879935.100149:0:2399:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100152:0:2399:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879935.100156:0:2399:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f62000. 02000000:00000001:1.0:1779879935.100158:0:2399:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879935.100164:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879935.100168:0:2399:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba5d8. 00000400:00000010:1.0:1779879935.100174:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac3b8. 00000100:00000200:1.0:1779879935.100180:0:2399:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080448, portal 25 00000100:00000001:1.0:1779879935.100182:0:2399:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879935.100183:0:2399:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195624448 : -113665513927168 : ffff989f2f6b4e00) 00000100:00000040:1.0:1779879935.100188:0:2399:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100194:0:2399:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879935.100198:0:2399:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879935.100204:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac5d8. 00000100:00000200:1.0:1779879935.100207:0:2399:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1866328745080448, offset 0 00000400:00000200:1.0:1779879935.100214:0:2399:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100222:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100228:0:2399:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879935.100233:0:2399:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879935.100237:0:2399:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879935.100240:0:2399:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879935.100243:0:2399:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100246:0:2399:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547896:547896:256:4294967295] 192.168.203.137@tcp LPNI seq info [547835:547835:8:4294967295] 00000400:00000200:1.0:1779879935.100251:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100259:0:2399:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879935.100265:0:2399:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879935.100269:0:2399:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f0a08e800. 00000800:00000200:1.0:1779879935.100276:0:2399:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879935.100283:0:2399:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879935.100287:0:2399:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f0a08e800 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879935.100297:0:2399:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879935.100301:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879935.100303:0:2399:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 0 is 4305009349 00000100:00000010:1.0:1779879935.100307:0:2399:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334840. 00000100:00000001:1.0:1779879935.100313:0:2399:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 0 00000100:00000001:1.0:1779879935.100318:0:2399:0:(niobuf.c:663:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS new deadline estimation for cpu 1 is 4305009349 00000100:00000010:1.0:1779879935.100321:0:2399:0:(niobuf.c:666:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at ffff989f1e334240. 00000100:00000001:1.0:1779879935.100323:0:2399:0:(niobuf.c:678:kick_cpu_latency()) MGC192.168.203.137@tcp: PM QoS request now active for cpu 1 00000100:00000001:1.0:1779879935.100326:0:2399:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879935.100329:0:2399:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100332:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100334:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879935.100383:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879935.100387:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f0a08e800. 00000400:00000200:1.0:1779879935.100397:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879935.100402:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879935.100407:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac5d8 00000400:00000010:1.0:1779879935.100409:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac5d8. 00000100:00000001:1.0:1779879935.100413:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879935.100416:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100424:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879935.100427:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100433:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100435:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000010:1.0:1779879935.100447:0:214361:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b4000. 00000020:00000040:1.0:1779879935.100450:0:214361:0:(genops.c:1231:class_import_get()) import ffff989f08d17800 refcount=9 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879935.100452:0:214361:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879935.100454:0:214361:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879935.100456:0:214361:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879935.100459:0:214361:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f0a08e800. 00000100:00000001:1.0:1779879935.100463:0:214361:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100467:0:214361:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0000_UUID req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879935.100474:0:214361:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879935.100476:0:214361:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100479:0:214361:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000a81a7720] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100491:0:214361:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100494:0:214361:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879935.100497:0:214361:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1779879935.100502:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.100504:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.100505:0:2399:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879935.100508:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100520:0:2399:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879935.100522:0:2399:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100524:0:2399:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879935.100526:0:2399:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879935.100527:0:2399:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879935.100530:0:2399:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080576:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879935.100533:0:2399:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879935.100536:0:2399:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879935.100537:0:2399:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100539:0:2399:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879935.100541:0:2399:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f63c00. 02000000:00000001:1.0:1779879935.100543:0:2399:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879935.100547:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879935.100549:0:2399:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07ebac60. 00000400:00000010:1.0:1779879935.100553:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac5d8. 00000100:00000200:1.0:1779879935.100556:0:2399:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080576, portal 10 00000100:00000001:1.0:1779879935.100558:0:2399:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879935.100559:0:2399:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195620864 : -113665513930752 : ffff989f2f6b4000) 00000100:00000040:1.0:1779879935.100565:0:2399:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100571:0:2399:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879935.100574:0:2399:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879935.100576:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9acf68. 00000100:00000200:1.0:1779879935.100578:0:2399:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745080576, offset 0 00000400:00000200:1.0:1779879935.100582:0:2399:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100586:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100589:0:2399:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879935.100592:0:2399:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879935.100594:0:2399:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879935.100596:0:2399:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879935.100598:0:2399:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100606:0:2399:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547897:547897:256:4294967295] 192.168.203.137@tcp LPNI seq info [547836:547836:8:4294967295] 00000400:00000200:1.0:1779879935.100611:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100617:0:2399:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879935.100621:0:2399:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879935.100624:0:2399:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f0a08e300. 00000800:00000200:1.0:1779879935.100628:0:2399:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879935.100633:0:2399:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879935.100636:0:2399:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f0a08e300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879935.100643:0:2399:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100647:0:2399:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: using a timeout of 16940 usecs (17 jiffies) 00000400:00000200:1.0:1779879935.100650:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879935.100652:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305005585 00000100:00000001:1.0:1779879935.100655:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.100657:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305005585 00000100:00000001:1.0:1779879935.100659:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0000-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.100661:0:2399:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879935.100663:0:2399:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100705:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100708:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.100712:0:2399:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879935.100713:0:2399:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1779879935.100715:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.100716:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.100718:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100719:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879935.100736:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879935.100738:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f0a08e300. 00000400:00000200:1.0:1779879935.100741:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879935.100745:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879935.100748:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9acf68 00000400:00000010:1.0:1779879935.100750:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9acf68. 00000100:00000001:1.0:1779879935.100758:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879935.100762:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100769:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879935.100771:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100776:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.100777:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000010:1.0:1779879935.100800:0:214361:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b4700. 00000020:00000040:1.0:1779879935.100803:0:214361:0:(genops.c:1231:class_import_get()) import ffff989f08d17000 refcount=4 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000100:00000001:1.0:1779879935.100805:0:214361:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879935.100807:0:214361:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879935.100808:0:214361:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879935.100812:0:214361:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f0a08e300. 00000100:00000001:1.0:1779879935.100816:0:214361:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100820:0:214361:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-MDT0001_UUID req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879935.100827:0:214361:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879935.100829:0:214361:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100832:0:214361:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000b5998ac5] to pc [ptlrpcd_00_00+0] req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100845:0:214361:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.100847:0:214361:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:1.0:1779879935.100851:0:214361:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000001:1.0:1779879935.100856:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.100858:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.100859:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879935.100861:0:2399:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00000001:1.0:1779879935.100863:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.100866:0:2399:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879935.100867:0:2399:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.100869:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.100875:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.100877:0:2399:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879935.100886:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100894:0:2399:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879935.100895:0:2399:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100897:0:2399:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879935.100899:0:2399:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879935.100901:0:2399:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879935.100904:0:2399:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080704:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879935.100907:0:2399:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879935.100909:0:2399:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879935.100910:0:2399:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.100913:0:2399:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879935.100916:0:2399:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f62200. 02000000:00000001:1.0:1779879935.100918:0:2399:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879935.100921:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879935.100923:0:2399:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba1b8. 00000400:00000010:1.0:1779879935.100927:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9acf68. 00000100:00000200:1.0:1779879935.100930:0:2399:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080704, portal 10 00000100:00000001:1.0:1779879935.100932:0:2399:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879935.100933:0:2399:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195622656 : -113665513928960 : ffff989f2f6b4700) 00000100:00000040:1.0:1779879935.100939:0:2399:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.100945:0:2399:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879935.100947:0:2399:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879935.100949:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9accc0. 00000100:00000200:1.0:1779879935.100952:0:2399:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1866328745080704, offset 0 00000400:00000200:1.0:1779879935.100956:0:2399:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100959:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100963:0:2399:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879935.100966:0:2399:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879935.100969:0:2399:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879935.100975:0:2399:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879935.100978:0:2399:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879935.100981:0:2399:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547898:547898:256:4294967295] 192.168.203.137@tcp LPNI seq info [547837:547837:8:4294967295] 00000400:00000200:1.0:1779879935.100985:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.100992:0:2399:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879935.100996:0:2399:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879935.101000:0:2399:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8f00. 00000800:00000200:1.0:1779879935.101004:0:2399:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879935.101008:0:2399:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879935.101011:0:2399:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8f00 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879935.101018:0:2399:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.101021:0:2399:0:(niobuf.c:635:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: using a timeout of 24457 usecs (25 jiffies) 00000400:00000200:1.0:1779879935.101025:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879935.101027:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305005594 00000100:00000001:1.0:1779879935.101029:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.101031:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305005594 00000100:00000001:1.0:1779879935.101033:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-MDT0001-mdc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.101034:0:2399:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879935.101036:0:2399:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101039:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101041:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879935.101070:0:2390:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879935.101073:0:2390:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8f00. 00000400:00000200:1.0:1779879935.101076:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879935.101079:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879935.101082:0:2390:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9accc0 00000400:00000010:1.0:1779879935.101084:0:2390:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9accc0. 00000100:00000001:1.0:1779879935.101086:0:2390:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879935.101090:0:2390:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101108:0:2390:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879935.101111:0:2390:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101117:0:2390:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101118:0:2390:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000010:1.0:1779879935.101128:0:214361:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at ffff989f2f6b7100. 00000020:00000040:1.0:1779879935.101130:0:214361:0:(genops.c:1231:class_import_get()) import ffff989f08d11800 refcount=4 obd=lustre-OST0000-osc-ffff989f2f61e800 00000100:00000001:1.0:1779879935.101133:0:214361:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:1.0:1779879935.101134:0:214361:0:(sec.c:443:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:1.0:1779879935.101136:0:214361:0:(sec.c:467:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:1.0:1779879935.101140:0:214361:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at ffff989f12aa8f00. 00000100:00000001:1.0:1779879935.101143:0:214361:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.101147:0:214361:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0000_UUID req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:1.0:1779879935.101155:0:214361:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:1.0:1779879935.101157:0:214361:0:(jobid.c:976:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.101160:0:214361:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [0000000013cb743e] to pc [ptlrpcd_00_01+1] req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101172:0:214361:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.101175:0:214361:0:(pinger.c:231:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000040:1.0:1779879935.101178:0:214361:0:(pinger.c:249:ptlrpc_pinger_process_import()) 1e19ffe7-b601-47d3-a8bd-54e8e6c5db34->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00000001:1.0:1779879935.101186:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.101187:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.101189:0:2399:0:(client.c:1799:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1779879935.101192:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101199:0:2399:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1779879935.101200:0:2399:0:(client.c:1422:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.101202:0:2399:0:(sec.c:718:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1779879935.101210:0:2399:0:(sec.c:749:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1779879935.101212:0:2399:0:(sec.c:793:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1779879935.101217:0:2399:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff989f2f6b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080832:192.168.203.137@tcp:400:kworker.0 00000100:00000001:1.0:1779879935.101219:0:2399:0:(niobuf.c:866:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1779879935.101221:0:2399:0:(sec.c:1089:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1779879935.101223:0:2399:0:(sec.c:1126:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1779879935.101225:0:2399:0:(sec.c:1873:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1779879935.101227:0:2399:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at ffff989f35f63a00. 02000000:00000001:1.0:1779879935.101229:0:2399:0:(sec.c:1883:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1779879935.101233:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000010:1.0:1779879935.101235:0:2399:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at ffff989f07eba840. 00000400:00000010:1.0:1779879935.101239:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9accc0. 00000100:00000200:1.0:1779879935.101242:0:2399:0:(niobuf.c:1097:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1866328745080832, portal 4 00000100:00000001:1.0:1779879935.101244:0:2399:0:(client.c:3265:ptlrpc_request_addref()) Process entered 00000100:00000001:1.0:1779879935.101245:0:2399:0:(client.c:3267:ptlrpc_request_addref()) Process leaving (rc=18446630408195633408 : -113665513918208 : ffff989f2f6b7100) 00000100:00000040:1.0:1779879935.101251:0:2399:0:(niobuf.c:1117:ptl_send_rpc()) @@@ send flags=200 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101258:0:2399:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1779879935.101260:0:2399:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.137@tcp 00000400:00000010:1.0:1779879935.101263:0:2399:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at ffff989f1d9ac110. 00000100:00000200:1.0:1779879935.101265:0:2399:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1866328745080832, offset 0 00000400:00000200:1.0:1779879935.101269:0:2399:0:(lib-move.c:5065:LNetPut()) LNetPut -> 12345-192.168.203.137@tcp 00000400:00000200:1.0:1779879935.101272:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.101276:0:2399:0:(lib-move.c:2716:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.137@tcp local destination 00000400:00000200:1.0:1779879935.101278:0:2399:0:(lib-move.c:2346:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:1.0:1779879935.101281:0:2399:0:(lib-move.c:1532:lnet_get_best_ni()) selected best_ni 192.168.203.37@tcp 00000400:00000200:1.0:1779879935.101283:0:2399:0:(lib-move.c:1086:lnet_select_peer_ni()) 192.168.203.37@tcp lpni_is_preferred = 0 00000400:00000200:1.0:1779879935.101285:0:2399:0:(lib-move.c:1155:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.137@tcp 00000400:00000200:1.0:1779879935.101294:0:2399:0:(lib-move.c:1652:lnet_handle_send()) 192.168.203.37@tcp NI seq info: [547899:547899:256:4294967295] 192.168.203.137@tcp LPNI seq info [547838:547838:8:4294967295] 00000400:00000200:1.0:1779879935.101298:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:1.0:1779879935.101304:0:2399:0:(lib-move.c:1767:lnet_handle_send()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp:) -> 192.168.203.137@tcp(192.168.203.137@tcp:192.168.203.137@tcp) : PUT try# 0 00000800:00000200:1.0:1779879935.101308:0:2399:0:(socklnd_cb.c:931:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.137@tcp 00000800:00000010:1.0:1779879935.101316:0:2399:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at ffff989f12aa8300. 00000800:00000200:1.0:1779879935.101320:0:2399:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [ffff989f08d53000] -> 12345-192.168.203.137@tcp (5) 00000800:00000200:1.0:1779879935.101324:0:2399:0:(socklnd_cb.c:678:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.137@tcp ip 192.168.203.137:988 00000800:00000200:1.0:1779879935.101327:0:2399:0:(socklnd_cb.c:697:ksocknal_queue_tx_locked()) Packet ffff989f12aa8300 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1779879935.101334:0:2399:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1779879935.101337:0:2399:0:(niobuf.c:635:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: using a timeout of 83296 usecs (84 jiffies) 00000400:00000200:1.0:1779879935.101340:0:2399:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000100:00000001:1.0:1779879935.101342:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 0 is 4305005653 00000100:00000001:1.0:1779879935.101345:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 0, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.101347:0:2399:0:(niobuf.c:663:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS new deadline estimation for cpu 1 is 4305005653 00000100:00000001:1.0:1779879935.101348:0:2399:0:(niobuf.c:692:kick_cpu_latency()) lustre-OST0000-osc-ffff989f2f61e800: PM QoS request active for cpu 1, keep current deadline 4305009349 00000100:00000001:1.0:1779879935.101350:0:2399:0:(niobuf.c:1135:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:1.0:1779879935.101352:0:2399:0:(client.c:1927:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101354:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101356:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:1.0:1779879935.101359:0:2399:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:1.0:1779879935.101360:0:2399:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1779879935.101362:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1779879935.101363:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1779879935.101365:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101367:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:1.0:1779879935.101387:0:2391:0:(socklnd_cb.c:469:ksocknal_process_transmit()) send(0) 0 00000800:00000010:1.0:1779879935.101390:0:2391:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at ffff989f12aa8300. 00000400:00000200:1.0:1779879935.101392:0:2391:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:1.0:1779879935.101396:0:2391:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000400:00000200:1.0:1779879935.101399:0:2391:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac110 00000400:00000010:1.0:1779879935.101401:0:2391:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac110. 00000100:00000001:1.0:1779879935.101404:0:2391:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1779879935.101408:0:2391:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101414:0:2391:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:1.0:1779879935.101417:0:2391:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 1 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1779879935.101428:0:2391:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1779879935.101430:0:2391:0:(events.c:68:request_out_callback()) Process leaving 00000100:00000040:1.0:1779879935.101438:0:214361:0:(pinger.c:306:ptlrpc_pinger_main()) next wakeup in 5 (10343) 00000100:00000001:0.0:1779879935.101883:0:2398:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879935.101887:0:2398:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:0.0:1779879935.108311:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108321:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1779879935.108325:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108327:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108335:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108342:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:0.0:1779879935.108369:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108374:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108379:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108383:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27700 00000400:00000200:0.0:1779879935.108388:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108406:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fb01 [1] + 224 00000800:00000001:0.0:1779879935.108414:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108419:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108422:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1779879935.108426:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:0.0:1779879935.108433:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108454:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1779879935.108464:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108479:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1779879935.108499:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108509:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1779879935.108512:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108514:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108517:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108521:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:0.0:1779879935.108525:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108528:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108530:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108533:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 25 MB=0x6a16a85e27680 00000400:00000200:0.0:1779879935.108536:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108540:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53faf1 [1] + 224 00000800:00000001:0.0:1779879935.108544:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108547:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108549:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1779879935.108552:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:0.0:1779879935.108555:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108559:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1779879935.108566:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108579:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1779879935.108584:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108588:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1779879935.108590:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108591:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108595:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108598:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:0.0:1779879935.108602:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108604:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108607:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108609:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 10 MB=0x6a16a85e27780 00000400:00000200:0.0:1779879935.108612:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108615:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fb11 [1] + 224 00000800:00000001:0.0:1779879935.108619:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108621:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108623:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1779879935.108626:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:0.0:1779879935.108629:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108633:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1779879935.108641:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108648:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1779879935.108652:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108655:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:0.0:1779879935.108657:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108659:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108662:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108665:0:2390:0:(lib-move.c:4556:lnet_parse()) TRACE: 192.168.203.37@tcp(192.168.203.37@tcp) <- 192.168.203.137@tcp : PUT - for me 00000400:00000200:0.0:1779879935.108698:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108700:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108714:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108717:0:2390:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.137@tcp of length 224 into portal 4 MB=0x6a16a85e27800 00000400:00000200:0.0:1779879935.108720:0:2390:0:(api-ni.c:1558:lnet_nid4_cpt_hash()) Match nid 192.168.203.137@tcp to cpt 0 00000400:00000200:0.0:1779879935.108724:0:2390:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.137@tcp of length 224/224 into md 0x53fb21 [1] + 224 00000800:00000001:0.0:1779879935.108728:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108734:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:0.0:1779879935.108736:0:2390:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:0.0:1779879935.108738:0:2390:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.37@tcp->192.168.203.137@tcp: PUT: OK 00000100:00000001:0.0:1779879935.108742:0:2390:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108746:0:2390:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1779879935.108753:0:2390:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108760:0:2390:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:0.0:1779879935.108764:0:2390:0:(socklnd_cb.c:281:ksocknal_receive()) Process entered 00000800:00000001:0.0:1779879935.108766:0:2390:0:(socklnd_cb.c:314:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.108777:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879935.108780:0:2399:0:(client.c:1976:ptlrpc_check_set()) Process entered 00000400:00000001:0.0:1779879935.108785:0:2399:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879935.108797:0:2399:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879935.108800:0:2399:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba840. 00000400:00000200:0.0:1779879935.108804:0:2399:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9accc0 00000400:00000010:0.0:1779879935.108806:0:2399:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9accc0. 00000100:00000001:0.0:1779879935.108809:0:2399:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108813:0:2399:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879935.108821:0:2399:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108827:0:2399:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879935.108829:0:2399:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.108831:0:2399:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879935.108835:0:2399:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879935.108841:0:2399:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879935.108843:0:2399:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879935.108846:0:2399:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879935.108855:0:2399:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f08d11c60 time=139 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879935.108865:0:2399:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d11c18 time=96 v=5 (1 1 0 0) 00000100:00000001:0.0:1779879935.108868:0:2399:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879935.108870:0:2399:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.108872:0:2399:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879935.108875:0:2399:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879935.108879:0:2399:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879935.108881:0:2399:0:(client.c:3098:ptlrpc_free_committed()) lustre-OST0000-osc-ffff989f2f61e800: skip recheck: last_committed 68719497264 00000100:00000001:0.0:1779879935.108884:0:2399:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879935.108885:0:2399:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879935.108890:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108898:0:2399:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879935.108901:0:2399:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879935.108903:0:2399:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879935.108907:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879935.108914:0:2399:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080832:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879935.108920:0:2399:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879935.108922:0:2399:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879935.108923:0:2399:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba1b8. 00000400:00000200:0.0:1779879935.108926:0:2399:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9acf68 00000400:00000010:0.0:1779879935.108927:0:2399:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9acf68. 00000100:00000001:0.0:1779879935.108929:0:2399:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.108932:0:2399:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879935.108940:0:2399:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.108946:0:2399:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879935.108954:0:2399:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.108956:0:2399:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879935.108957:0:2399:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879935.108958:0:2399:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879935.108960:0:2399:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879935.108961:0:2399:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879935.108966:0:2399:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17460 time=139 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879935.108969:0:2399:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17418 time=26 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879935.108971:0:2399:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879935.108972:0:2399:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.108974:0:2399:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00080000:0.0:1779879935.108978:0:2399:0:(ldlm_request.c:1663:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=46162) req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00010000:00000001:0.0:1779879935.108984:0:2399:0:(ldlm_request.c:1664:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879935.108986:0:2399:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879935.108994:0:2399:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0001-mdc-ffff989f2f61e800: skip recheck: last_committed 21474836545 00000100:00000001:0.0:1779879935.108995:0:2399:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879935.108997:0:2399:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879935.109000:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109007:0:2399:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879935.109009:0:2399:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879935.109010:0:2399:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879935.109013:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879935.109020:0:2399:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080704:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879935.109025:0:2399:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879935.109026:0:2399:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879935.109027:0:2399:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07ebac60. 00000400:00000200:0.0:1779879935.109030:0:2399:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac5d8 00000400:00000010:0.0:1779879935.109032:0:2399:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac5d8. 00000100:00000001:0.0:1779879935.109034:0:2399:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.109043:0:2399:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879935.109051:0:2399:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109057:0:2399:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879935.109058:0:2399:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109060:0:2399:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879935.109061:0:2399:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879935.109062:0:2399:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879935.109064:0:2399:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879935.109066:0:2399:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879935.109070:0:2399:0:(import.c:2151:obd_at_measure()) add 60 to ffff989f08d17c60 time=139 v=60 (60 60 60 60) 00000100:00001000:0.0:1779879935.109073:0:2399:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f08d17c18 time=48 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879935.109075:0:2399:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879935.109077:0:2399:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.109078:0:2399:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879935.109080:0:2399:0:(ldlm_request.c:1675:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879935.109082:0:2399:0:(client.c:3086:ptlrpc_free_committed()) Process entered 00000100:00000040:0.0:1779879935.109083:0:2399:0:(client.c:3098:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff989f2f61e800: skip recheck: last_committed 30064771623 00000100:00000001:0.0:1779879935.109085:0:2399:0:(client.c:3099:ptlrpc_free_committed()) Process leaving 00000100:00000001:0.0:1779879935.109086:0:2399:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879935.109090:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109096:0:2399:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879935.109098:0:2399:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879935.109099:0:2399:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879935.109102:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879935.109109:0:2399:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:1e19ffe7-b601-47d3-a8bd-54e8e6c5db34:2399:1866328745080576:192.168.203.137@tcp:400:kworker.0 00000400:00000001:0.0:1779879935.109114:0:2399:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:0.0:1779879935.109115:0:2399:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1779879935.109345:0:2399:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at ffff989f07eba5d8. 00000400:00000200:0.0:1779879935.109348:0:2399:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md ffff989f1d9ac3b8 00000400:00000010:0.0:1779879935.109350:0:2399:0:(lib-lnet.h:276:lnet_md_free()) slab-freed 'md': 136 at ffff989f1d9ac3b8. 00000100:00000001:0.0:1779879935.109353:0:2399:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:0.0:1779879935.109367:0:2399:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000200:0.0:1779879935.109375:0:2399:0:(events.c:102:reply_in_callback()) @@@ unlink req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109382:0:2399:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:0.0:1779879935.109383:0:2399:0:(client.c:2978:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109385:0:2399:0:(client.c:1579:after_reply()) Process entered 02000000:00000001:0.0:1779879935.109387:0:2399:0:(sec.c:1134:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1779879935.109388:0:2399:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1779879935.109390:0:2399:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1779879935.109392:0:2399:0:(sec.c:1190:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:0.0:1779879935.109395:0:2399:0:(import.c:2151:obd_at_measure()) add 5 to ffff989f02475460 time=139 v=5 (5 5 5 5) 00000100:00001000:0.0:1779879935.109397:0:2399:0:(import.c:2151:obd_at_measure()) add 1 to ffff989f02475418 time=48 v=5 (1 1 1 1) 00000100:00000001:0.0:1779879935.109399:0:2399:0:(client.c:1481:ptlrpc_check_status()) Process entered 00000100:00000001:0.0:1779879935.109400:0:2399:0:(client.c:1519:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.109402:0:2399:0:(ldlm_request.c:1645:ldlm_cli_update_pool()) Process entered 00010000:00000001:0.0:1779879935.109404:0:2399:0:(ldlm_request.c:1649:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879935.109406:0:2399:0:(client.c:1783:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1779879935.109409:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109415:0:2399:0:(client.c:2331:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1779879935.109417:0:2399:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:0.0:1779879935.109419:0:2399:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:0.0:1779879935.109422:0:2399:0:(lustre_net.h:2432:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1779879935.109429:0:2399:0:(client.c:2391:ptlrpc_check_set()) Completed RPC req@ffff989f2f6b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:93850f6c-6c43-4b8a-8d90-a4353df8a1d5:2399:1866328745080448:192.168.203.137@tcp:400:kworker.0 00000100:00000001:0.0:1779879935.109433:0:2399:0:(client.c:2442:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109436:0:2399:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879935.109444:0:2399:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b7100 x1866328745080832/t0(0) o400->lustre-OST0000-osc-ffff989f2f61e800@192.168.203.137@tcp:28/4 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109451:0:2399:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879935.109454:0:2399:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879935.109457:0:2399:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f63a00. 02000000:00000001:0.0:1779879935.109460:0:2399:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879935.109463:0:2399:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879935.109465:0:2399:0:(genops.c:1244:class_import_put()) import ffff989f08d11800 refcount=3 obd=lustre-OST0000-osc-ffff989f2f61e800 00000020:00000001:0.0:1779879935.109467:0:2399:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879935.109469:0:2399:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f12aa8f00. 02000000:00000001:0.0:1779879935.109473:0:2399:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879935.109474:0:2399:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879935.109476:0:2399:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b7100. 00000100:00000001:0.0:1779879935.109479:0:2399:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879935.109481:0:2399:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109493:0:2399:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879935.109497:0:2399:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b4700 x1866328745080704/t0(0) o400->lustre-MDT0001-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109818:0:2399:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879935.109820:0:2399:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879935.109822:0:2399:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f62200. 02000000:00000001:0.0:1779879935.109826:0:2399:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879935.109827:0:2399:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879935.109828:0:2399:0:(genops.c:1244:class_import_put()) import ffff989f08d17000 refcount=3 obd=lustre-MDT0001-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879935.109830:0:2399:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879935.109831:0:2399:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f0a08e300. 02000000:00000001:0.0:1779879935.109834:0:2399:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879935.109835:0:2399:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879935.109837:0:2399:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b4700. 00000100:00000001:0.0:1779879935.109838:0:2399:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879935.109840:0:2399:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109842:0:2399:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879935.109846:0:2399:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b4000 x1866328745080576/t0(0) o400->lustre-MDT0000-mdc-ffff989f2f61e800@192.168.203.137@tcp:12/10 lens 224/224 e 0 to 0 dl 1779880020 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109852:0:2399:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879935.109859:0:2399:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879935.109861:0:2399:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f63c00. 02000000:00000001:0.0:1779879935.109863:0:2399:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879935.109865:0:2399:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879935.109866:0:2399:0:(genops.c:1244:class_import_put()) import ffff989f08d17800 refcount=8 obd=lustre-MDT0000-mdc-ffff989f2f61e800 00000020:00000001:0.0:1779879935.109868:0:2399:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879935.109869:0:2399:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f0a08e800. 02000000:00000001:0.0:1779879935.109871:0:2399:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879935.109872:0:2399:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879935.109874:0:2399:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b4000. 00000100:00000001:0.0:1779879935.109876:0:2399:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879935.109877:0:2399:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109879:0:2399:0:(client.c:2844:__ptlrpc_req_put()) Process entered 00000100:00000040:0.0:1779879935.109883:0:2399:0:(client.c:2852:__ptlrpc_req_put()) @@@ refcount now 0 req@ffff989f2f6b4e00 x1866328745080448/t0(0) o400->MGC192.168.203.137@tcp@192.168.203.137@tcp:26/25 lens 224/224 e 0 to 0 dl 1779879951 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1779879935.109888:0:2399:0:(client.c:2770:__ptlrpc_free_req()) Process entered 02000000:00000001:0.0:1779879935.109889:0:2399:0:(sec.c:1898:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1779879935.109890:0:2399:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at ffff989f35f62000. 02000000:00000001:0.0:1779879935.109892:0:2399:0:(sec.c:1912:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1779879935.109893:0:2399:0:(genops.c:1238:class_import_put()) Process entered 00000020:00000040:0.0:1779879935.109894:0:2399:0:(genops.c:1244:class_import_put()) import ffff989f02475000 refcount=3 obd=MGC192.168.203.137@tcp 00000020:00000001:0.0:1779879935.109896:0:2399:0:(genops.c:1251:class_import_put()) Process leaving 02000000:00000010:0.0:1779879935.109897:0:2399:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at ffff989f0a08eb00. 02000000:00000001:0.0:1779879935.109898:0:2399:0:(sec.c:483:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:0.0:1779879935.109900:0:2399:0:(sec.c:500:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:0.0:1779879935.109901:0:2399:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at ffff989f2f6b4e00. 00000100:00000001:0.0:1779879935.109903:0:2399:0:(client.c:2826:__ptlrpc_free_req()) Process leaving 00000100:00000001:0.0:1779879935.109904:0:2399:0:(client.c:2885:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109906:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1779879935.109914:0:2399:0:(client.c:2616:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1779879935.109916:0:2399:0:(client.c:2644:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1779879935.109918:0:2399:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1779879935.109921:0:2399:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.548747:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879935.548752:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879935.548758:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879935.548760:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:1.0:1779879935.548958:0:245969:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:1.0:1779879935.548962:0:245969:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:1.0:1779879935.548965:0:245969:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00000001:0.0:1779879936.571901:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879936.571922:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879936.571929:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879936.571932:0:287881:0:(ldlm_request.c:2313:ldlm_cancel_lru()) Process entered 00010000:00000001:0.0:1779879936.571934:0:287881:0:(ldlm_request.c:2101:ldlm_prepare_lru_list()) Process entered 00010000:00000001:0.0:1779879936.571940:0:287881:0:(ldlm_lock.c:185:ldlm_lock_put()) Process entered 00010000:00000001:0.0:1779879936.571942:0:287881:0:(ldlm_lock.c:224:ldlm_lock_put()) Process leaving 00010000:00000001:0.0:1779879936.571943:0:287881:0:(ldlm_request.c:2273:ldlm_prepare_lru_list()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879936.571946:0:287881:0:(ldlm_lockd.c:2241:ldlm_bl_to_thread()) Process entered 00010000:00000001:0.0:1779879936.571948:0:287881:0:(ldlm_lockd.c:2244:ldlm_bl_to_thread()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879936.571950:0:287881:0:(ldlm_request.c:2323:ldlm_cancel_lru()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879936.571951:0:287881:0:(ldlm_pool.c:497:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1779879936.575063:0:239871:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1779879936.575069:0:239871:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1779879936.575073:0:239871:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1779879936.727958:0:288269:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000001:00000001:0.0:1779879937.012350:0:288305:0:(debug.c:650:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:0.0:1779879937.012355:0:288305:0:(debug.c:651:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity test_816: @@@@@@ FAIL: client: import is not in IDLE state after 50 00000001:00000001:0.0:1779879937.032033:0:288305:0:(debug.c:653:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:0.0:1779879937.597741:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879937.597746:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:0.0:1779879937.597753:0:287881:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:0.0:1779879937.597754:0:287881:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1779879937.597905:0:251676:0:(genops.c:1916:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1779879937.597909:0:251676:0:(genops.c:1930:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1779879937.597913:0:251676:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1779879938.158003:0:288405:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) Debug log: 1956 lines, 1956 kept, 0 dropped, 0 bad.