00010000:00000001:2.0F:1773941874.993577:0:25480:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941874.993594:0:25480:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0F:1773941874.993701:0:36408:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1773941874.993725:0:36408:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1773941874.993730:0:36408:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:2.0:1773941875.119647:0:36804:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1773941876.016371:0:25480:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941876.016378:0:25480:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:2.0:1773941876.016399:0:36409:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:2.0:1773941876.016402:0:36409:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:2.0:1773941876.016404:0:36409:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000100:3.0F:1773941876.017616:0:34759:0:(lib-socket.c:453:lnet_sock_connect()) Error -113 connecting 1023 -> 192.168.203.150:988 00000400:00000100:3.0:1773941876.017733:0:34759:0:(acceptor.c:111:lnet_connect_console_error()) Connection to 192.168.203.150@tcp at host 192.168.203.150 was unreachable: the network or that node may be down, or Lustre may be misconfigured. 00000400:00000200:3.0:1773941876.017745:0:34759:0:(router.c:1750:lnet_notify()) 192.168.203.40@tcp notifying 192.168.203.150@tcp: down 00000800:00000200:3.0:1773941876.017753:0:34759:0:(socklnd.c:1681:ksocknal_notify_gw_down()) gw 192.168.203.150@tcp down 00000800:00000100:3.0:1773941876.017762:0:34759:0:(socklnd_cb.c:422:ksocknal_txlist_done()) Deleting packet type 2 len 0 0@<0:0>->192.168.203.150@tcp 00000800:00000010:3.0:1773941876.017765:0:34759:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 232 at 0000000067550678. 00000400:00000200:3.0:1773941876.017771:0:34759:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = -113, hstatus = 8 00000400:00000200:3.0:1773941876.017775:0:34759:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.150@tcp: GET: REMOTE_DROPPED 00000400:00000200:3.0:1773941876.017781:0:34759:0:(peer.c:3081:lnet_discovery_event_handler()) Received event: 5 00000400:00000200:3.0:1773941876.017783:0:34759:0:(peer.c:3033:lnet_discovery_event_send()) Ping Send to 192.168.203.150@tcp: 1 00000400:00000200:3.0:1773941876.017787:0:34759:0:(peer.c:2405:lnet_peer_queue_for_discovery()) Queue peer 192.168.203.150@tcp: -114 00000400:00000200:2.0:1773941876.017818:0:34768:0:(peer.c:3988:lnet_peer_discovery_wait_for_work()) woken: 0 00000400:00000200:2.0:1773941876.017825:0:34768:0:(peer.c:4100:lnet_peer_discovery()) peer 192.168.203.150@tcp(0000000029592723) state 0x841 00000400:00000001:2.0:1773941876.017832:0:34768:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000001:2.0:1773941876.017834:0:34768:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000200:2.0:1773941876.017837:0:34768:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000c6ff3842 00000400:00000010:2.0:1773941876.017839:0:34768:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 00000000c6ff3842. 00000400:00000200:2.0:1773941876.017843:0:34768:0:(peer.c:3081:lnet_discovery_event_handler()) Received event: 6 00000400:00000200:2.0:1773941876.017846:0:34768:0:(peer.c:2405:lnet_peer_queue_for_discovery()) Queue peer 192.168.203.150@tcp: -114 00000400:00000010:2.0:1773941876.017849:0:34768:0:(api-ni.c:1839:lnet_ping_buffer_free()) kfreed 'pbuf': 281 at 0000000088bb3f9a. 00000400:00000200:2.0:1773941876.017853:0:34768:0:(peer.c:3747:lnet_peer_ping_failed()) peer 192.168.203.150@tcp:-113 00000400:00000200:2.0:1773941876.017856:0:34768:0:(peer.c:4118:lnet_peer_discovery()) peer 192.168.203.150@tcp(0000000029592723) state 0x2041 rc -113 00000400:00000200:2.0:1773941876.017859:0:34768:0:(peer.c:2421:lnet_peer_discovery_complete()) Discovery complete. Dequeue peer 192.168.203.150@tcp 00000100:00000040:3.0:1773941876.208422:0:26380:0:(connection.c:75:cpu_latency_work()) work item of 0000000013aa02ea (cpu 0) has reached its deadline 4295307403, at 4295307456 00000100:00000040:2.0:1773941876.208423:0:25480:0:(connection.c:75:cpu_latency_work()) work item of 00000000983dcb09 (cpu 0) has reached its deadline 4295307403, at 4295307456 00000100:00000040:1.0F:1773941876.208423:0:811:0:(connection.c:75:cpu_latency_work()) work item of 00000000402f81b0 (cpu 0) has reached its deadline 4295307403, at 4295307456 00000100:00000040:0.0:1773941876.208423:0:429:0:(connection.c:75:cpu_latency_work()) work item of 000000006233c58e (cpu 0) has reached its deadline 4295307403, at 4295307456 00000100:00000010:3.0:1773941876.208455:0:26380:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 000000005d01c377. 00000100:00000010:2.0:1773941876.208457:0:25480:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 00000000f0938459. 00000100:00000010:1.0:1773941876.208466:0:811:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000061ff68b1. 00000100:00000010:0.0:1773941876.208469:0:429:0:(connection.c:94:cpu_latency_work()) kfreed 'pm_qos_req_done': 72 at 0000000021973743. 00010000:00000001:2.0:1773941877.040389:0:25480:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941877.040394:0:25480:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1773941877.040435:0:36408:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1773941877.040439:0:36408:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1773941877.040442:0:36408:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:0.0:1773941877.383518:0:36998:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000080:00000001:1.0:1773941877.761695:0:37089:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00000001:00000001:0.0:1773941877.889961:0:37125:0:(debug.c:644:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:0.0:1773941877.889965:0:37125:0:(debug.c:645:libcfs_debug_mark_buffer()) DEBUG MARKER: == conf-sanity test 162: replace nids with -o noclient === 13:37:57 (1773941877) 00000001:00000001:0.0:1773941877.892381:0:37125:0:(debug.c:647:libcfs_debug_mark_buffer()) ************************************************** 00010000:00000001:2.0:1773941878.064399:0:25480:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941878.064405:0:25480:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:0.0:1773941878.064432:0:36409:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:0.0:1773941878.064436:0:36409:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:0.0:1773941878.064440:0:36409:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00000040:0.0:1773941878.064774:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) a4a01d0f-f05b-4ab9-85a6-d00edfad63d8->MGS: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1773941878.064779:0:8:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1773941878.064786:0:8:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000ba621f44. 00000020:00000040:0.0:1773941878.064791:0:8:0:(genops.c:1229:class_import_get()) import 0000000005b35796 refcount=4 obd=MGC192.168.203.140@tcp 00000100:00000001:0.0:1773941878.064795:0:8:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1773941878.064797:0:8:0:(sec.c:440:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1773941878.064802:0:8:0:(sec.c:464:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1773941878.064809:0:8:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 0000000013b81800. 00000100:00000001:0.0:1773941878.064815:0:8:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064824:0:8:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging a4a01d0f-f05b-4ab9-85a6-d00edfad63d8->MGS req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1773941878.064833:0:8:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1773941878.064834:0:8:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064845:0:8:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000ba621f44] to pc [ptlrpcd_00_00+0] req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1773941878.064880:0:8:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064882:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6d068363-87d2-4c60-86c5-ab4d9650de7d->lustre-MDT0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1773941878.064885:0:8:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1773941878.064888:0:8:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 000000003f460552. 00000020:00000040:0.0:1773941878.064891:0:8:0:(genops.c:1229:class_import_get()) import 000000003350b0b5 refcount=4 obd=lustre-MDT0000-mdc-ffffa0da908d9000 00000100:00000001:0.0:1773941878.064901:0:8:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1773941878.064902:0:8:0:(sec.c:440:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1773941878.064904:0:8:0:(sec.c:464:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1773941878.064907:0:8:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 00000000fd73a78d. 00000100:00000001:0.0:1773941878.064910:0:8:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064914:0:8:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 6d068363-87d2-4c60-86c5-ab4d9650de7d->lustre-MDT0000_UUID req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1773941878.064920:0:8:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1773941878.064921:0:8:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064924:0:8:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [000000003f460552] to pc [ptlrpcd_00_01+1] req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1773941878.064932:0:8:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064934:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 6d068363-87d2-4c60-86c5-ab4d9650de7d->lustre-OST0000_UUID: level FULL/9 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00000001:0.0:1773941878.064936:0:8:0:(pinger.c:119:ptlrpc_ping()) Process entered 00000100:00000010:0.0:1773941878.064940:0:8:0:(client.c:592:ptlrpc_request_cache_alloc()) slab-alloced 'req': 888 at 00000000e87286be. 00000020:00000040:0.0:1773941878.064942:0:8:0:(genops.c:1229:class_import_get()) import 000000003ea07add refcount=3 obd=lustre-OST0000-osc-ffffa0da908d9000 00000100:00000001:0.0:1773941878.064943:0:8:0:(client.c:844:ptlrpc_request_bufs_pack()) Process entered 02000000:00000001:0.0:1773941878.064944:0:8:0:(sec.c:440:sptlrpc_req_get_ctx()) Process entered 02000000:00000001:0.0:1773941878.064946:0:8:0:(sec.c:464:sptlrpc_req_get_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:0.0:1773941878.064949:0:8:0:(sec_null.c:146:null_alloc_reqbuf()) kmalloced '(req->rq_reqbuf)': 256 at 000000005ceaadd3. 00000100:00000001:0.0:1773941878.064951:0:8:0:(client.c:924:ptlrpc_request_bufs_pack()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064953:0:8:0:(pinger.c:133:ptlrpc_ping()) @@@ pinging 6d068363-87d2-4c60-86c5-ab4d9650de7d->lustre-OST0000_UUID req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00000001:0.0:1773941878.064958:0:8:0:(jobid.c:936:lustre_get_jobid()) Process entered 00000100:00000001:0.0:1773941878.064959:0:8:0:(jobid.c:947:lustre_get_jobid()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.064962:0:8:0:(ptlrpcd.c:235:ptlrpcd_add_req()) @@@ add req [00000000e87286be] to pc [ptlrpcd_00_02+2] req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.064984:0:34770:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1773941878.064990:0:8:0:(pinger.c:142:ptlrpc_ping()) Process leaving (rc=0 : 0 : 0) 00000008:00000010:3.0:1773941878.064994:0:34770:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2856 at 000000002778f610. 00000100:00000040:0.0:1773941878.064994:0:8:0:(pinger.c:302:ptlrpc_pinger_main()) next wakeup in 5 (647) 00020000:00000010:3.0:1773941878.065002:0:34770:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 0000000006e310da. 00000100:00000001:0.0:1773941878.065006:0:34771:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000080:00000010:3.0:1773941878.065007:0:34770:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000095531d9b. 00000080:00000010:3.0:1773941878.065013:0:34770:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 584 at 000000006ce5840d. 00000008:00000010:0.0:1773941878.065014:0:34771:0:(osc_dev.c:88:osc_key_init()) slab-alloced 'info': 2856 at 000000001cdf1647. 00000008:00000010:3.0:1773941878.065017:0:34770:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 0000000020b27616. 00020000:00000010:3.0:1773941878.065021:0:34770:0:(lov_dev.c:116:lov_session_key_init()) slab-alloced 'info': 576 at 00000000a4a81fc3. 00020000:00000010:0.0:1773941878.065026:0:34771:0:(lov_dev.c:78:lov_key_init()) slab-alloced 'info': 192 at 0000000031d074d3. 00000080:00000010:3.0:1773941878.065028:0:34770:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 00000000949d7e3c. 00000100:00000001:3.0:1773941878.065032:0:34770:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000080:00000010:0.0:1773941878.065032:0:34771:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 0000000001062477. 00000100:00000001:3.0:1773941878.065034:0:34770:0:(client.c:1797:ptlrpc_send_new_req()) Process entered 00000080:00000010:0.0:1773941878.065037:0:34771:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 584 at 00000000bdcdfdf4. 00000100:00000040:3.0:1773941878.065041:0:34770:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000008:00000010:0.0:1773941878.065043:0:34771:0:(osc_dev.c:115:osc_session_init()) slab-alloced 'info': 352 at 00000000d3450a7d. 00020000:00000010:0.0:1773941878.065050:0:34771:0:(lov_dev.c:116:lov_session_key_init()) slab-alloced 'info': 576 at 0000000060323616. 00000100:00000001:3.0:1773941878.065052:0:34770:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:3.0:1773941878.065055:0:34770:0:(client.c:1421:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:0.0:1773941878.065056:0:34771:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 000000009298b125. 02000000:00000001:3.0:1773941878.065058:0:34770:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:3.0:1773941878.065062:0:34770:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1773941878.065063:0:34771:0:(client.c:1974:ptlrpc_check_set()) Process entered 02000000:00000001:3.0:1773941878.065066:0:34770:0:(sec.c:774:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.065067:0:34771:0:(client.c:1797:ptlrpc_send_new_req()) Process entered 00000100:00000040:0.0:1773941878.065073:0:34771:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1773941878.065078:0:34770:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@00000000ba621f44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a4a01d0f-f05b-4ab9-85a6-d00edfad63d8:34770:1860112843020928:192.168.203.140@tcp:400: 00000100:00000001:3.0:1773941878.065085:0:34770:0:(niobuf.c:865:ptl_send_rpc()) Process entered 00000100:00000001:0.0:1773941878.065091:0:34771:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 02000000:00000001:3.0:1773941878.065093:0:34770:0:(sec.c:1068:sptlrpc_cli_wrap_request()) Process entered 00000100:00000001:0.0:1773941878.065093:0:34771:0:(client.c:1421:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1773941878.065099:0:34770:0:(sec.c:1105:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1773941878.065099:0:34771:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:0.0:1773941878.065103:0:34771:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1773941878.065105:0:34770:0:(sec.c:1852:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000001:0.0:1773941878.065109:0:34771:0:(sec.c:774:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 02000000:00000010:3.0:1773941878.065113:0:34770:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000a5db5138. 00000100:00100000:0.0:1773941878.065117:0:34771:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@000000003f460552 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6d068363-87d2-4c60-86c5-ab4d9650de7d:34771:1860112843021056:192.168.203.140@tcp:400: 02000000:00000001:3.0:1773941878.065118:0:34770:0:(sec.c:1862:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.065123:0:34771:0:(niobuf.c:865:ptl_send_rpc()) Process entered 02000000:00000001:0.0:1773941878.065127:0:34771:0:(sec.c:1068:sptlrpc_cli_wrap_request()) Process entered 00000400:00000010:3.0:1773941878.065140:0:34770:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at 0000000096ad5d13. 02000000:00000001:0.0:1773941878.065142:0:34771:0:(sec.c:1105:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1773941878.065146:0:34771:0:(sec.c:1852:sptlrpc_cli_alloc_repbuf()) Process entered 00000400:00000010:3.0:1773941878.065151:0:34770:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000caf43100. 02000000:00000010:0.0:1773941878.065151:0:34771:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 00000000d0b5231a. 02000000:00000001:0.0:1773941878.065154:0:34771:0:(sec.c:1862:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000400:00000010:0.0:1773941878.065159:0:34771:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at 00000000b619bea6. 00000100:00000200:3.0:1773941878.065162:0:34770:0:(niobuf.c:1096:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1860112843020928, portal 25 00000400:00000010:0.0:1773941878.065163:0:34771:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000bc69d436. 00000100:00000001:3.0:1773941878.065183:0:34770:0:(client.c:3244:ptlrpc_request_addref()) Process entered 00000100:00000200:0.0:1773941878.065186:0:34771:0:(niobuf.c:1096:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1860112843021056, portal 10 00000100:00000001:3.0:1773941878.065187:0:34770:0:(client.c:3246:ptlrpc_request_addref()) Process leaving (rc=18446639459456237568 : -104614253314048 : ffffa0da9897c000) 00000100:00000001:0.0:1773941878.065189:0:34771:0:(client.c:3244:ptlrpc_request_addref()) Process entered 00000100:00000001:0.0:1773941878.065192:0:34771:0:(client.c:3246:ptlrpc_request_addref()) Process leaving (rc=18446639459456244736 : -104614253306880 : ffffa0da9897dc00) 00000020:00000010:0.0:1773941878.065199:0:34771:0:(lprocfs_status.c:1408:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000975226f5. 00000100:00000040:3.0:1773941878.065200:0:34770:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:0.0:1773941878.065210:0:34771:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.065220:0:34770:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000001:0.0:1773941878.065225:0:34771:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:3.0:1773941878.065230:0:34770:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.140@tcp 00000100:00000040:0.0:1773941878.065231:0:34771:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.140@tcp 00000400:00000010:3.0:1773941878.065237:0:34770:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 000000001480394a. 00000100:00000200:3.0:1773941878.065243:0:34770:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 26, xid 1860112843020928, offset 0 00000400:00000010:0.0:1773941878.065244:0:34771:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000087e1a570. 00000100:00000200:0.0:1773941878.065250:0:34771:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 12, xid 1860112843021056, offset 0 00000400:00000200:3.0:1773941878.065258:0:34770:0:(lib-move.c:5316:LNetPut()) LNetPut -> 12345-192.168.203.140@tcp 00000400:00000200:0.0:1773941878.065267:0:34771:0:(lib-move.c:5316:LNetPut()) LNetPut -> 12345-192.168.203.140@tcp 00000400:00000200:3.0:1773941878.065275:0:34770:0:(lib-move.c:2973:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.140@tcp local destination 00000400:00000200:3.0:1773941878.065280:0:34770:0:(lib-move.c:2603:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:3.0:1773941878.065284:0:34770:0:(lib-move.c:1789:lnet_get_best_ni()) selected best_ni 192.168.203.40@tcp 00000400:00000200:3.0:1773941878.065288:0:34770:0:(lib-move.c:1343:lnet_select_peer_ni()) 192.168.203.40@tcp lpni_is_preferred = 0 00000400:00000200:3.0:1773941878.065291:0:34770:0:(lib-move.c:1412:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.140@tcp 00000400:00000200:3.0:1773941878.065295:0:34770:0:(lib-move.c:1909:lnet_handle_send()) 192.168.203.40@tcp NI seq info: [57:57:256:4294967295] 192.168.203.140@tcp LPNI seq info [56:56:8:4294967295] 00000400:00000200:3.0:1773941878.065306:0:34770:0:(lib-move.c:2024:lnet_handle_send()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp:) -> 192.168.203.140@tcp(192.168.203.140@tcp:192.168.203.140@tcp) : PUT try# 0 00000800:00000200:3.0:1773941878.065312:0:34770:0:(socklnd_cb.c:1001:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.140@tcp 00000800:00000010:3.0:1773941878.065317:0:34770:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000713dfc0b. 00000400:00000200:0.0:1773941878.065329:0:34771:0:(lib-move.c:2973:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.140@tcp local destination 00000400:00000200:0.0:1773941878.065332:0:34771:0:(lib-move.c:2603:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000400:00000200:0.0:1773941878.065336:0:34771:0:(lib-move.c:1789:lnet_get_best_ni()) selected best_ni 192.168.203.40@tcp 00000400:00000200:0.0:1773941878.065340:0:34771:0:(lib-move.c:1343:lnet_select_peer_ni()) 192.168.203.40@tcp lpni_is_preferred = 0 00000400:00000200:0.0:1773941878.065343:0:34771:0:(lib-move.c:1412:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.140@tcp 00000400:00000200:0.0:1773941878.065348:0:34771:0:(lib-move.c:1909:lnet_handle_send()) 192.168.203.40@tcp NI seq info: [58:58:255:4294967295] 192.168.203.140@tcp LPNI seq info [57:57:7:4294967295] 00000800:00000200:3.0:1773941878.065353:0:34770:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000bca86f5c] -> 12345-192.168.203.140@tcp (5) 00000800:00000200:3.0:1773941878.065366:0:34770:0:(socklnd_cb.c:748:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.140@tcp ip 192.168.203.140:988 00000400:00000200:0.0:1773941878.065368:0:34771:0:(lib-move.c:2024:lnet_handle_send()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp:) -> 192.168.203.140@tcp(192.168.203.140@tcp:192.168.203.140@tcp) : PUT try# 0 00000800:00000200:3.0:1773941878.065373:0:34770:0:(socklnd_cb.c:767:ksocknal_queue_tx_locked()) Packet 00000000713dfc0b type 1, nob 320 niov 1 nkiov 1 00000800:00000200:0.0:1773941878.065378:0:34771:0:(socklnd_cb.c:1001:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.140@tcp 00000800:00000010:0.0:1773941878.065382:0:34771:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000c3669e0b. 00000100:00000001:3.0:1773941878.065394:0:34770:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:0.0:1773941878.065398:0:34771:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000bca86f5c] -> 12345-192.168.203.140@tcp (5) 00000100:00000001:3.0:1773941878.065401:0:34770:0:(niobuf.c:662:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS new deadline estimation for cpu 0 is 4295312461 00000800:00000200:0.0:1773941878.065407:0:34771:0:(socklnd_cb.c:748:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.140@tcp ip 192.168.203.140:988 00000100:00000010:3.0:1773941878.065409:0:34770:0:(niobuf.c:665:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 0000000063183f7a. 00000800:00000200:0.0:1773941878.065411:0:34771:0:(socklnd_cb.c:767:ksocknal_queue_tx_locked()) Packet 00000000c3669e0b type 1, nob 320 niov 1 nkiov 1 00000100:00000001:3.0:1773941878.065417:0:34770:0:(niobuf.c:677:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS request now active for cpu 0 00000100:00000001:0.0:1773941878.065417:0:34771:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:0.0:1773941878.065424:0:34771:0:(niobuf.c:634:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: using a timeout of 4420 usecs (5 jiffies) 00000100:00000001:3.0:1773941878.065425:0:34770:0:(niobuf.c:662:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS new deadline estimation for cpu 1 is 4295312461 00000100:00000001:0.0:1773941878.065429:0:34771:0:(niobuf.c:662:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 0 is 4295309318 00000100:00000010:3.0:1773941878.065431:0:34770:0:(niobuf.c:665:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d968d190. 00000100:00000001:0.0:1773941878.065452:0:34771:0:(niobuf.c:691:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS request active for cpu 0, keep current deadline 4295312461 00000100:00000001:3.0:1773941878.065453:0:34770:0:(niobuf.c:677:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS request now active for cpu 1 00000100:00000001:0.0:1773941878.065457:0:34771:0:(niobuf.c:662:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 1 is 4295309318 00000100:00000001:3.0:1773941878.065459:0:34770:0:(niobuf.c:662:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS new deadline estimation for cpu 2 is 4295312461 00000100:00000001:0.0:1773941878.065460:0:34771:0:(niobuf.c:691:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS request active for cpu 1, keep current deadline 4295312461 00000100:00000010:3.0:1773941878.065464:0:34770:0:(niobuf.c:665:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 00000000d2aa2b7b. 00000100:00000001:0.0:1773941878.065465:0:34771:0:(niobuf.c:662:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 2 is 4295309318 00000100:00000001:3.0:1773941878.065471:0:34770:0:(niobuf.c:677:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS request now active for cpu 2 00000100:00000001:3.0:1773941878.065474:0:34770:0:(niobuf.c:662:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS new deadline estimation for cpu 3 is 4295312461 00000100:00000001:0.0:1773941878.065475:0:34771:0:(niobuf.c:691:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS request active for cpu 2, keep current deadline 4295312461 00000100:00000010:3.0:1773941878.065477:0:34770:0:(niobuf.c:665:kick_cpu_latency()) kmalloced '(latency_qos->pm_qos_req)': 72 at 000000003a9c367b. 00000100:00000001:0.0:1773941878.065478:0:34771:0:(niobuf.c:662:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 3 is 4295309318 00000100:00000001:3.0:1773941878.065483:0:34770:0:(niobuf.c:677:kick_cpu_latency()) MGC192.168.203.140@tcp: PM QoS request now active for cpu 3 00000100:00000001:3.0:1773941878.065489:0:34770:0:(niobuf.c:1134:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1773941878.065489:0:34771:0:(niobuf.c:691:kick_cpu_latency()) lustre-MDT0000-mdc-ffffa0da908d9000: PM QoS request active for cpu 3, keep current deadline 4295312461 00000100:00000001:3.0:1773941878.065496:0:34770:0:(client.c:1925:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.065496:0:34771:0:(niobuf.c:1134:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000100:00000001:0.0:1773941878.065502:0:34771:0:(client.c:1925:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065503:0:34770:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065509:0:34770:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1773941878.065509:0:34771:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065514:0:34770:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:3.0:1773941878.065516:0:34770:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:3.0:1773941878.065520:0:34770:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1773941878.065522:0:34771:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.065523:0:34770:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1773941878.065526:0:34770:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065530:0:34770:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.065531:0:34771:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1773941878.065534:0:34771:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:0.0:1773941878.065538:0:34771:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1773941878.065540:0:34773:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1773941878.065541:0:34771:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1773941878.065543:0:34772:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000080:00000010:3.0:1773941878.065545:0:34773:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000f7c68237. 00000100:00000001:0.0:1773941878.065545:0:34771:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:1.0:1773941878.065550:0:34772:0:(vvp_dev.c:68:ll_thread_key_init()) slab-alloced 'lti': 168 at 00000000d4bd11cc. 00000080:00000010:3.0:1773941878.065552:0:34773:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 584 at 000000008d4e41f0. 00000100:00000001:0.0:1773941878.065552:0:34771:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000080:00000010:1.0:1773941878.065556:0:34772:0:(vvp_dev.c:119:vvp_thread_key_init()) slab-alloced 'vti': 584 at 00000000a8f3c61d. 00000080:00000010:3.0:1773941878.065559:0:34773:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 0000000058a0cc5d. 00000080:00000010:1.0:1773941878.065563:0:34772:0:(vvp_dev.c:94:vvp_session_key_init()) slab-alloced 'session': 240 at 0000000009309a59. 00000100:00000001:3.0:1773941878.065564:0:34773:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1773941878.065569:0:34772:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1773941878.065571:0:34772:0:(client.c:1797:ptlrpc_send_new_req()) Process entered 00000100:00000040:1.0:1773941878.065576:0:34772:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from New to Rpc req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 0 ref 1 fl New:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1773941878.065583:0:34772:0:(client.c:1363:ptlrpc_import_delay_req()) Process entered 00000100:00000001:1.0:1773941878.065585:0:34772:0:(client.c:1421:ptlrpc_import_delay_req()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1773941878.065587:0:34772:0:(sec.c:715:sptlrpc_req_refresh_ctx()) Process entered 02000000:00000001:1.0:1773941878.065589:0:34772:0:(sec.c:746:sptlrpc_req_refresh_ctx()) Process leaving via out_sec_put (rc=0 : 0 : 0x0) 02000000:00000001:1.0:1773941878.065591:0:34772:0:(sec.c:774:sptlrpc_req_refresh_ctx()) Process leaving (rc=0 : 0 : 0) 00000100:00100000:1.0:1773941878.065593:0:34772:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@00000000e87286be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6d068363-87d2-4c60-86c5-ab4d9650de7d:34772:1860112843021184:192.168.203.140@tcp:400: 00000100:00000001:1.0:1773941878.065596:0:34772:0:(niobuf.c:865:ptl_send_rpc()) Process entered 02000000:00000001:1.0:1773941878.065599:0:34772:0:(sec.c:1068:sptlrpc_cli_wrap_request()) Process entered 02000000:00000001:1.0:1773941878.065600:0:34772:0:(sec.c:1105:sptlrpc_cli_wrap_request()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:1.0:1773941878.065602:0:34772:0:(sec.c:1852:sptlrpc_cli_alloc_repbuf()) Process entered 02000000:00000010:1.0:1773941878.065605:0:34772:0:(sec_null.c:189:null_alloc_repbuf()) kmalloced '(req->rq_repbuf)': 512 at 000000000734646e. 02000000:00000001:1.0:1773941878.065607:0:34772:0:(sec.c:1862:sptlrpc_cli_alloc_repbuf()) Process leaving (rc=0 : 0 : 0) 00000800:00000200:3.0:1773941878.065609:0:34764:0:(socklnd_cb.c:539:ksocknal_process_transmit()) send(0) 0 00000400:00000010:1.0:1773941878.065610:0:34772:0:(lib-me.c:70:LNetMEAttach()) slab-alloced 'me': 88 at 000000002c575116. 00000800:00000010:3.0:1773941878.065617:0:34764:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000713dfc0b. 00000400:00000010:1.0:1773941878.065617:0:34772:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 00000000be949fe1. 00000100:00000200:1.0:1773941878.065624:0:34772:0:(niobuf.c:1096:ptl_send_rpc()) Setup reply buffer: 512 bytes, xid 1860112843021184, portal 4 00000100:00000001:1.0:1773941878.065625:0:34772:0:(client.c:3244:ptlrpc_request_addref()) Process entered 00000400:00000200:3.0:1773941878.065626:0:34764:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1773941878.065627:0:34772:0:(client.c:3246:ptlrpc_request_addref()) Process leaving (rc=18446639459456250112 : -104614253301504 : ffffa0da9897f100) 00000020:00000010:1.0:1773941878.065634:0:34772:0:(lprocfs_status.c:1408:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000e6ac6415. 00000400:00000200:3.0:1773941878.065639:0:34764:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000100:00000040:1.0:1773941878.065643:0:34772:0:(niobuf.c:1116:ptl_send_rpc()) @@@ send flags=200 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:3.0:1773941878.065647:0:34764:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 000000001480394a 00000100:00000001:1.0:1773941878.065649:0:34772:0:(niobuf.c:69:ptl_send_buf()) Process entered 00000100:00000040:1.0:1773941878.065651:0:34772:0:(niobuf.c:72:ptl_send_buf()) peer_id 12345-192.168.203.140@tcp 00000400:00000010:3.0:1773941878.065656:0:34764:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 000000001480394a. 00000400:00000010:1.0:1773941878.065660:0:34772:0:(lib-md.c:131:lnet_md_build()) slab-alloced 'lmd': 136 at 0000000050567b0b. 00000100:00000001:3.0:1773941878.065662:0:34764:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:1.0:1773941878.065664:0:34772:0:(niobuf.c:94:ptl_send_buf()) Sending 224 bytes to portal 28, xid 1860112843021184, offset 0 00000100:00000200:3.0:1773941878.065668:0:34764:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1773941878.065673:0:34772:0:(lib-move.c:5316:LNetPut()) LNetPut -> 12345-192.168.203.140@tcp 00000400:00000200:1.0:1773941878.065680:0:34772:0:(lib-move.c:2973:lnet_handle_send_case_locked()) Source ANY to MR: 192.168.203.140@tcp local destination 00000400:00000200:1.0:1773941878.065685:0:34772:0:(lib-move.c:2603:lnet_find_best_ni_on_local_net()) selected best_lpn tcp 00000100:00000001:3.0:1773941878.065688:0:34764:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000400:00000200:1.0:1773941878.065688:0:34772:0:(lib-move.c:1789:lnet_get_best_ni()) selected best_ni 192.168.203.40@tcp 00000400:00000200:1.0:1773941878.065693:0:34772:0:(lib-move.c:1343:lnet_select_peer_ni()) 192.168.203.40@tcp lpni_is_preferred = 0 00000100:00000040:3.0:1773941878.065696:0:34764:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:1.0:1773941878.065696:0:34772:0:(lib-move.c:1412:lnet_select_peer_ni()) sd_best_lpni = 192.168.203.140@tcp 00000400:00000200:1.0:1773941878.065701:0:34772:0:(lib-move.c:1909:lnet_handle_send()) 192.168.203.40@tcp NI seq info: [59:59:254:4294967295] 192.168.203.140@tcp LPNI seq info [58:58:6:4294967295] 00000100:00000001:3.0:1773941878.065717:0:34764:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000400:00000200:1.0:1773941878.065717:0:34772:0:(lib-move.c:2024:lnet_handle_send()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp:) -> 192.168.203.140@tcp(192.168.203.140@tcp:192.168.203.140@tcp) : PUT try# 0 00000100:00000001:3.0:1773941878.065723:0:34764:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:1.0:1773941878.065726:0:34772:0:(socklnd_cb.c:1001:ksocknal_send()) sending 224 bytes in 1 frags to 12345-192.168.203.140@tcp 00000800:00000010:1.0:1773941878.065730:0:34772:0:(socklnd_cb.c:41:ksocknal_alloc_tx()) alloc '(tx)': 248 at 00000000955e6c86. 00000800:00000200:1.0:1773941878.065734:0:34772:0:(socklnd.c:214:ksocknal_find_peer_locked()) got peer_ni [00000000bca86f5c] -> 12345-192.168.203.140@tcp (5) 00000800:00000200:1.0:1773941878.065745:0:34772:0:(socklnd_cb.c:748:ksocknal_queue_tx_locked()) Sending to 12345-192.168.203.140@tcp ip 192.168.203.140:988 00000800:00000200:1.0:1773941878.065747:0:34772:0:(socklnd_cb.c:767:ksocknal_queue_tx_locked()) Packet 00000000955e6c86 type 1, nob 320 niov 1 nkiov 1 00000100:00000001:1.0:1773941878.065750:0:34772:0:(niobuf.c:111:ptl_send_buf()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:1.0:1773941878.065752:0:34772:0:(niobuf.c:634:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: using a timeout of 3232 usecs (4 jiffies) 00000800:00000200:3.0:1773941878.065754:0:34764:0:(socklnd_cb.c:539:ksocknal_process_transmit()) send(0) 0 00000100:00000001:1.0:1773941878.065755:0:34772:0:(niobuf.c:662:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 0 is 4295309317 00000800:00000010:3.0:1773941878.065758:0:34764:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000c3669e0b. 00000100:00000001:1.0:1773941878.065759:0:34772:0:(niobuf.c:691:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS request active for cpu 0, keep current deadline 4295312461 00000100:00000001:1.0:1773941878.065764:0:34772:0:(niobuf.c:662:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 1 is 4295309317 00000400:00000200:3.0:1773941878.065766:0:34764:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:1.0:1773941878.065766:0:34772:0:(niobuf.c:691:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS request active for cpu 1, keep current deadline 4295312461 00000100:00000001:1.0:1773941878.065767:0:34772:0:(niobuf.c:662:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 2 is 4295309317 00000100:00000001:1.0:1773941878.065771:0:34772:0:(niobuf.c:691:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS request active for cpu 2, keep current deadline 4295312461 00000400:00000200:3.0:1773941878.065776:0:34764:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000100:00000001:1.0:1773941878.065776:0:34772:0:(niobuf.c:662:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS new deadline estimation for cpu 3 is 4295309317 00000100:00000001:1.0:1773941878.065782:0:34772:0:(niobuf.c:691:kick_cpu_latency()) lustre-OST0000-osc-ffffa0da908d9000: PM QoS request active for cpu 3, keep current deadline 4295312461 00000400:00000200:3.0:1773941878.065783:0:34764:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000087e1a570 00000100:00000001:1.0:1773941878.065784:0:34772:0:(niobuf.c:1134:ptl_send_rpc()) Process leaving via out (rc=0 : 0 : 0x0) 00000400:00000010:3.0:1773941878.065786:0:34764:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 0000000087e1a570. 00000100:00000001:1.0:1773941878.065787:0:34772:0:(client.c:1925:ptlrpc_send_new_req()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065790:0:34764:0:(events.c:34:request_out_callback()) Process entered 00000100:00000001:1.0:1773941878.065791:0:34772:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000200:3.0:1773941878.065794:0:34764:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1773941878.065801:0:34772:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.065806:0:34764:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000100:00000001:1.0:1773941878.065808:0:34772:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:3.0:1773941878.065811:0:34764:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:1.0:1773941878.065811:0:34772:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=11 : 11 : b) 00000100:00000001:1.0:1773941878.065816:0:34772:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:1.0:1773941878.065819:0:34772:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000001:1.0:1773941878.065822:0:34772:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065823:0:34764:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:1.0:1773941878.065825:0:34772:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065826:0:34764:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000200:3.0:1773941878.065865:0:34764:0:(socklnd_cb.c:539:ksocknal_process_transmit()) send(0) 0 00000800:00000010:3.0:1773941878.065868:0:34764:0:(socklnd_cb.c:98:ksocknal_free_tx()) kfreed 'tx': 248 at 00000000955e6c86. 00000400:00000200:3.0:1773941878.065870:0:34764:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:3.0:1773941878.065872:0:34764:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000400:00000200:3.0:1773941878.065874:0:34764:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 0000000050567b0b 00000400:00000010:3.0:1773941878.065875:0:34764:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 0000000050567b0b. 00000100:00000001:3.0:1773941878.065876:0:34764:0:(events.c:34:request_out_callback()) Process entered 00000100:00000200:3.0:1773941878.065878:0:34764:0:(events.c:39:request_out_callback()) @@@ type 5, status 0 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:Nr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.065884:0:34764:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000100:00000040:3.0:1773941878.065892:0:34764:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 1 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 2 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.065896:0:34764:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.065898:0:34764:0:(events.c:68:request_out_callback()) Process leaving 00000800:00000001:2.0:1773941878.067941:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.067952:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1773941878.067954:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.067957:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1773941878.067963:0:34763:0:(lib-move.c:4806:lnet_parse()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp) <- 192.168.203.140@tcp : PUT - for me 00000400:00000200:2.0:1773941878.067974:0:34763:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.140@tcp of length 224 into portal 10 MB=0x69bc345401b00 00000400:00000200:2.0:1773941878.067981:0:34763:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index a from 12345-192.168.203.140@tcp of length 224/224 into md 0x3bd [1] + 224 00000800:00000001:2.0:1773941878.067987:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.067992:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1773941878.067994:0:34763:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1773941878.068002:0:34763:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000100:00000001:2.0:1773941878.068006:0:34763:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000200:2.0:1773941878.068010:0:34763:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000040:2.0:1773941878.068018:0:34763:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1773941878.068036:0:34763:0:(events.c:163:reply_in_callback()) Process leaving 00000800:00000001:2.0:1773941878.068042:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.068046:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000800:00000001:2.0:1773941878.068047:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.068050:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.068052:0:34771:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000400:00000200:2.0:1773941878.068053:0:34763:0:(lib-move.c:4806:lnet_parse()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp) <- 192.168.203.140@tcp : PUT - for me 00000100:00000001:3.0:1773941878.068056:0:34771:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000400:00000001:3.0:1773941878.068063:0:34771:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1773941878.068063:0:34763:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.140@tcp of length 224 into portal 25 MB=0x69bc345401a80 00000400:00000001:3.0:1773941878.068067:0:34771:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:3.0:1773941878.068070:0:34771:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at 00000000b619bea6. 00000400:00000200:3.0:1773941878.068074:0:34771:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000bc69d436 00000400:00000010:3.0:1773941878.068075:0:34771:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 00000000bc69d436. 00000100:00000001:3.0:1773941878.068078:0:34771:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1773941878.068082:0:34763:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 19 from 12345-192.168.203.140@tcp of length 224/224 into md 0x3b9 [1] + 224 00000100:00000200:3.0:1773941878.068084:0:34771:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1773941878.068089:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.068095:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000200:3.0:1773941878.068097:0:34771:0:(events.c:102:reply_in_callback()) @@@ unlink req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000400:00000200:2.0:1773941878.068098:0:34763:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000400:00000200:2.0:1773941878.068108:0:34763:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000100:00000001:3.0:1773941878.068110:0:34771:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:2.0:1773941878.068111:0:34763:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:3.0:1773941878.068112:0:34771:0:(client.c:2957:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.068115:0:34771:0:(client.c:1578:after_reply()) Process entered 00000100:00000200:2.0:1773941878.068116:0:34763:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:3.0:1773941878.068118:0:34771:0:(sec.c:1113:do_cli_unwrap_reply()) Process entered 00000100:00000001:3.0:1773941878.068122:0:34771:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1773941878.068124:0:34771:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:3.0:1773941878.068127:0:34771:0:(sec.c:1169:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1773941878.068128:0:34763:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:3.0:1773941878.068149:0:34771:0:(import.c:2136:obd_at_measure()) add 5 to 0000000045163b0a time=5 v=5 (5 0 0 0) 00000100:00001000:3.0:1773941878.068153:0:34771:0:(import.c:2136:obd_at_measure()) add 1 to 000000004e4afe99 time=5 v=5 (1 0 0 0) 00000100:00000001:3.0:1773941878.068156:0:34771:0:(client.c:1480:ptlrpc_check_status()) Process entered 00000100:00000001:3.0:1773941878.068157:0:34771:0:(client.c:1518:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1773941878.068157:0:34763:0:(events.c:163:reply_in_callback()) Process leaving 00010000:00000001:3.0:1773941878.068161:0:34771:0:(ldlm_request.c:1516:ldlm_cli_update_pool()) Process entered 00000800:00000001:2.0:1773941878.068162:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00010000:00000001:3.0:1773941878.068164:0:34771:0:(ldlm_request.c:1546:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:3.0:1773941878.068184:0:34771:0:(client.c:3065:ptlrpc_free_committed()) Process entered 00000800:00000001:2.0:1773941878.068184:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:3.0:1773941878.068187:0:34771:0:(client.c:3077:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffffa0da908d9000: skip recheck: last_committed 0 00000800:00000001:2.0:1773941878.068188:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000100:00000001:3.0:1773941878.068193:0:34771:0:(client.c:3078:ptlrpc_free_committed()) Process leaving 00000800:00000001:2.0:1773941878.068195:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.068196:0:34771:0:(client.c:1781:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.068197:0:34770:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1773941878.068202:0:34770:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000040:3.0:1773941878.068208:0:34771:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000400:00000001:0.0:1773941878.068209:0:34770:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000400:00000200:2.0:1773941878.068211:0:34763:0:(lib-move.c:4806:lnet_parse()) TRACE: 192.168.203.40@tcp(192.168.203.40@tcp) <- 192.168.203.140@tcp : PUT - for me 00000400:00000001:0.0:1773941878.068214:0:34770:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:0.0:1773941878.068218:0:34770:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at 0000000096ad5d13. 00000400:00000200:0.0:1773941878.068225:0:34770:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000caf43100 00000400:00000200:2.0:1773941878.068228:0:34763:0:(lib-ptl.c:584:lnet_ptl_match_md()) Request from 12345-192.168.203.140@tcp of length 224 into portal 4 MB=0x69bc345401b80 00000400:00000010:0.0:1773941878.068230:0:34770:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 00000000caf43100. 00000100:00000001:0.0:1773941878.068239:0:34770:0:(events.c:78:reply_in_callback()) Process entered 00000400:00000200:2.0:1773941878.068242:0:34763:0:(lib-ptl.c:210:lnet_try_match_md()) Incoming put index 4 from 12345-192.168.203.140@tcp of length 224/224 into md 0x3c9 [1] + 224 00000100:00000200:0.0:1773941878.068247:0:34770:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000800:00000001:2.0:1773941878.068264:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 00000800:00000001:2.0:1773941878.068272:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1773941878.068278:0:34763:0:(lib-msg.c:1048:lnet_is_health_check()) health check = 1, status = 0, hstatus = 0 00000100:00000001:3.0:1773941878.068282:0:34771:0:(client.c:2329:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:3.0:1773941878.068287:0:34771:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000200:0.0:1773941878.068288:0:34770:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.068291:0:34771:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1773941878.068292:0:34763:0:(lib-msg.c:824:lnet_health_check()) health check: 192.168.203.40@tcp->192.168.203.140@tcp: PUT: OK 00000100:00000001:2.0:1773941878.068301:0:34763:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000040:3.0:1773941878.068304:0:34771:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1773941878.068307:0:34770:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000200:2.0:1773941878.068311:0:34763:0:(events.c:80:reply_in_callback()) @@@ type 2, status 0 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:0.0:1773941878.068312:0:34770:0:(client.c:2957:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00100000:3.0:1773941878.068349:0:34771:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@000000003f460552 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6d068363-87d2-4c60-86c5-ab4d9650de7d:34771:1860112843021056:192.168.203.140@tcp:400: 00000100:00000001:0.0:1773941878.068355:0:34770:0:(client.c:1578:after_reply()) Process entered 00000100:00000040:2.0:1773941878.068358:0:34763:0:(events.c:149:reply_in_callback()) @@@ reply in flags=200 mlen=224 offset=224 replen=224 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.068359:0:34771:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 02000000:00000001:0.0:1773941878.068360:0:34770:0:(sec.c:1113:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1773941878.068364:0:34770:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:3.0:1773941878.068366:0:34771:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000100:00000001:0.0:1773941878.068368:0:34770:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:0.0:1773941878.068375:0:34770:0:(sec.c:1169:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:3.0:1773941878.068376:0:34771:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@000000003f460552 x1860112843021056/t0(0) o400->lustre-MDT0000-mdc-ffffa0da908d9000@192.168.203.140@tcp:12/10 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00001000:0.0:1773941878.068384:0:34770:0:(import.c:2136:obd_at_measure()) add 5 to 000000000d441ba1 time=6 v=5 (5 0 0 0) 00000100:00001000:0.0:1773941878.068393:0:34770:0:(import.c:2136:obd_at_measure()) add 1 to 00000000e09d2c7a time=6 v=5 (1 0 0 0) 00000100:00000001:2.0:1773941878.068394:0:34763:0:(events.c:163:reply_in_callback()) Process leaving 00000100:00000001:3.0:1773941878.068395:0:34771:0:(client.c:2768:__ptlrpc_free_req()) Process entered 02000000:00000001:3.0:1773941878.068400:0:34771:0:(sec.c:1877:sptlrpc_cli_free_repbuf()) Process entered 00000100:00000001:0.0:1773941878.068401:0:34770:0:(client.c:1480:ptlrpc_check_status()) Process entered 00000800:00000001:2.0:1773941878.068403:0:34763:0:(socklnd_cb.c:345:ksocknal_receive()) Process entered 02000000:00000010:3.0:1773941878.068404:0:34771:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000d0b5231a. 00000100:00000001:0.0:1773941878.068405:0:34770:0:(client.c:1518:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00000800:00000001:2.0:1773941878.068408:0:34763:0:(socklnd_cb.c:383:ksocknal_receive()) Process leaving (rc=1 : 1 : 1) 00010000:00000001:0.0:1773941878.068411:0:34770:0:(ldlm_request.c:1516:ldlm_cli_update_pool()) Process entered 02000000:00000001:3.0:1773941878.068413:0:34771:0:(sec.c:1891:sptlrpc_cli_free_repbuf()) Process leaving 00010000:00000001:0.0:1773941878.068414:0:34770:0:(ldlm_request.c:1520:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1773941878.068433:0:34771:0:(genops.c:1236:class_import_put()) Process entered 00000100:00000001:0.0:1773941878.068433:0:34770:0:(client.c:1781:after_reply()) Process leaving (rc=0 : 0 : 0) 00000020:00000040:3.0:1773941878.068435:0:34771:0:(genops.c:1242:class_import_put()) import 000000003350b0b5 refcount=3 obd=lustre-MDT0000-mdc-ffffa0da908d9000 00000020:00000001:3.0:1773941878.068438:0:34771:0:(genops.c:1249:class_import_put()) Process leaving 00000100:00000040:0.0:1773941878.068439:0:34770:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 02000000:00000010:3.0:1773941878.068441:0:34771:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 00000000fd73a78d. 02000000:00000001:3.0:1773941878.068447:0:34771:0:(sec.c:480:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:0.0:1773941878.068448:0:34770:0:(client.c:2329:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 02000000:00000001:3.0:1773941878.068450:0:34771:0:(sec.c:497:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:3.0:1773941878.068451:0:34771:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 000000003f460552. 00000100:00000001:3.0:1773941878.068458:0:34771:0:(client.c:2824:__ptlrpc_free_req()) Process leaving 00000100:00000001:3.0:1773941878.068461:0:34771:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1773941878.068462:0:34770:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:3.0:1773941878.068464:0:34771:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:0.0:1773941878.068465:0:34770:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:3.0:1773941878.068469:0:34771:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000040:0.0:1773941878.068470:0:34770:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:3.0:1773941878.068472:0:34771:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1773941878.068476:0:34772:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:3.0:1773941878.068477:0:34771:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1773941878.068480:0:34772:0:(client.c:1974:ptlrpc_check_set()) Process entered 00000100:00000001:3.0:1773941878.068481:0:34771:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000400:00000001:2.0:1773941878.068487:0:34772:0:(lib-msg.c:22:lnet_build_unlink_event()) Process entered 00000100:00100000:0.0:1773941878.068487:0:34770:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@00000000ba621f44 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:a4a01d0f-f05b-4ab9-85a6-d00edfad63d8:34770:1860112843020928:192.168.203.140@tcp:400: 00000400:00000001:2.0:1773941878.068491:0:34772:0:(lib-msg.c:31:lnet_build_unlink_event()) Process leaving 00000400:00000010:2.0:1773941878.068494:0:34772:0:(lib-me.c:113:lnet_me_unlink()) slab-freed 'me': 88 at 000000002c575116. 00000100:00000001:0.0:1773941878.068495:0:34770:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000400:00000200:2.0:1773941878.068498:0:34772:0:(lib-md.c:46:lnet_md_unlink()) Unlinking md 00000000be949fe1 00000100:00000001:0.0:1773941878.068499:0:34770:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000400:00000010:2.0:1773941878.068503:0:34772:0:(lib-lnet.h:312:lnet_md_free()) slab-freed 'md': 136 at 00000000be949fe1. 00000100:00000040:0.0:1773941878.068505:0:34770:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000ba621f44 x1860112843020928/t0(0) o400->MGC192.168.203.140@tcp@192.168.203.140@tcp:26/25 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1773941878.068507:0:34772:0:(events.c:78:reply_in_callback()) Process entered 00000100:00000001:0.0:1773941878.068513:0:34770:0:(client.c:2768:__ptlrpc_free_req()) Process entered 00000100:00000200:2.0:1773941878.068514:0:34772:0:(events.c:80:reply_in_callback()) @@@ type 6, status 0 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQ/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1773941878.068516:0:34770:0:(sec.c:1877:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:0.0:1773941878.068520:0:34770:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 00000000a5db5138. 00000100:00000200:2.0:1773941878.068526:0:34772:0:(events.c:102:reply_in_callback()) @@@ unlink req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 02000000:00000001:0.0:1773941878.068526:0:34770:0:(sec.c:1891:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:0.0:1773941878.068528:0:34770:0:(genops.c:1236:class_import_put()) Process entered 00000020:00000040:0.0:1773941878.068530:0:34770:0:(genops.c:1242:class_import_put()) import 0000000005b35796 refcount=3 obd=MGC192.168.203.140@tcp 00000020:00000001:0.0:1773941878.068534:0:34770:0:(genops.c:1249:class_import_put()) Process leaving 02000000:00000010:0.0:1773941878.068537:0:34770:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 0000000013b81800. 02000000:00000001:0.0:1773941878.068542:0:34770:0:(sec.c:480:sptlrpc_req_put_ctx()) Process entered 00000100:00000001:2.0:1773941878.068543:0:34772:0:(events.c:163:reply_in_callback()) Process leaving 02000000:00000001:0.0:1773941878.068544:0:34770:0:(sec.c:497:sptlrpc_req_put_ctx()) Process leaving 00000100:00000001:2.0:1773941878.068546:0:34772:0:(client.c:2957:ptlrpc_unregister_reply()) Process leaving (rc=1 : 1 : 1) 00000100:00000010:0.0:1773941878.068547:0:34770:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000ba621f44. 00000100:00000001:2.0:1773941878.068551:0:34772:0:(client.c:1578:after_reply()) Process entered 00000100:00000001:0.0:1773941878.068553:0:34770:0:(client.c:2824:__ptlrpc_free_req()) Process leaving 02000000:00000001:2.0:1773941878.068555:0:34772:0:(sec.c:1113:do_cli_unwrap_reply()) Process entered 00000100:00000001:0.0:1773941878.068556:0:34770:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1773941878.068557:0:34772:0:(pack_generic.c:602:__lustre_unpack_msg()) Process entered 00000100:00000001:0.0:1773941878.068559:0:34770:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1773941878.068560:0:34772:0:(pack_generic.c:620:__lustre_unpack_msg()) Process leaving (rc=0 : 0 : 0) 02000000:00000001:2.0:1773941878.068563:0:34772:0:(sec.c:1169:do_cli_unwrap_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:0.0:1773941878.068564:0:34770:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:0.0:1773941878.068566:0:34770:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000020:00000010:2.0:1773941878.068569:0:34772:0:(lprocfs_status.c:1408:lprocfs_stats_alloc_one()) alloc '(stats->ls_percpu[cpuid])': 4608 at 00000000a1f44061. 00000100:00000001:0.0:1773941878.068569:0:34770:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:0.0:1773941878.068572:0:34770:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00000100:00001000:2.0:1773941878.068575:0:34772:0:(import.c:2136:obd_at_measure()) add 5 to 00000000f3681324 time=5 v=5 (5 0 0 0) 00000100:00001000:2.0:1773941878.068583:0:34772:0:(import.c:2136:obd_at_measure()) add 1 to 000000009f491d35 time=5 v=5 (1 0 0 0) 00000100:00000001:2.0:1773941878.068585:0:34772:0:(client.c:1480:ptlrpc_check_status()) Process entered 00000100:00000001:2.0:1773941878.068586:0:34772:0:(client.c:1518:ptlrpc_check_status()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1773941878.068588:0:34772:0:(ldlm_request.c:1516:ldlm_cli_update_pool()) Process entered 00010000:00080000:2.0:1773941878.068590:0:34772:0:(ldlm_request.c:1534:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92324) req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00010000:00000001:2.0:1773941878.068595:0:34772:0:(ldlm_request.c:1535:ldlm_cli_update_pool()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1773941878.068597:0:34772:0:(client.c:3065:ptlrpc_free_committed()) Process entered 00000100:00000040:2.0:1773941878.068598:0:34772:0:(client.c:3077:ptlrpc_free_committed()) lustre-OST0000-osc-ffffa0da908d9000: skip recheck: last_committed 0 00000100:00000001:2.0:1773941878.068599:0:34772:0:(client.c:3078:ptlrpc_free_committed()) Process leaving 00000100:00000001:2.0:1773941878.068600:0:34772:0:(client.c:1781:after_reply()) Process leaving (rc=0 : 0 : 0) 00000100:00000040:2.0:1773941878.068602:0:34772:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Rpc to Interpret req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1773941878.068606:0:34772:0:(client.c:2329:ptlrpc_check_set()) Process leaving via interpret (rc=0 : 0 : 0x0) 00000100:00000001:2.0:1773941878.068607:0:34772:0:(niobuf.c:473:ptlrpc_unregister_bulk()) Process entered 00000100:00000001:2.0:1773941878.068608:0:34772:0:(niobuf.c:487:ptlrpc_unregister_bulk()) Process leaving (rc=1 : 1 : 1) 00000100:00000040:2.0:1773941878.068610:0:34772:0:(lustre_net.h:2431:ptlrpc_rqphase_move()) @@@ move request phase from Interpret to Complete req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Interpret:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1773941878.068614:0:34772:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@00000000e87286be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6d068363-87d2-4c60-86c5-ab4d9650de7d:34772:1860112843021184:192.168.203.140@tcp:400: 00000100:00000001:2.0:1773941878.068617:0:34772:0:(client.c:2440:ptlrpc_check_set()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1773941878.068618:0:34772:0:(client.c:2842:__ptlrpc_req_put()) Process entered 00000100:00000040:2.0:1773941878.068619:0:34772:0:(client.c:2850:__ptlrpc_req_put()) @@@ refcount now 0 req@00000000e87286be x1860112843021184/t0(0) o400->lustre-OST0000-osc-ffffa0da908d9000@192.168.203.140@tcp:28/4 lens 224/224 e 0 to 0 dl 1773941894 ref 1 fl Complete:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00000001:2.0:1773941878.068624:0:34772:0:(client.c:2768:__ptlrpc_free_req()) Process entered 02000000:00000001:2.0:1773941878.068625:0:34772:0:(sec.c:1877:sptlrpc_cli_free_repbuf()) Process entered 02000000:00000010:2.0:1773941878.068626:0:34772:0:(sec_null.c:203:null_free_repbuf()) kfreed 'req->rq_repbuf': 512 at 000000000734646e. 02000000:00000001:2.0:1773941878.068628:0:34772:0:(sec.c:1891:sptlrpc_cli_free_repbuf()) Process leaving 00000020:00000001:2.0:1773941878.068629:0:34772:0:(genops.c:1236:class_import_put()) Process entered 00000020:00000040:2.0:1773941878.068630:0:34772:0:(genops.c:1242:class_import_put()) import 000000003ea07add refcount=2 obd=lustre-OST0000-osc-ffffa0da908d9000 00000020:00000001:2.0:1773941878.068631:0:34772:0:(genops.c:1249:class_import_put()) Process leaving 02000000:00000010:2.0:1773941878.068632:0:34772:0:(sec_null.c:173:null_free_reqbuf()) kvfree 'req->rq_reqbuf': 256 at 000000005ceaadd3. 02000000:00000001:2.0:1773941878.068639:0:34772:0:(sec.c:480:sptlrpc_req_put_ctx()) Process entered 02000000:00000001:2.0:1773941878.068640:0:34772:0:(sec.c:497:sptlrpc_req_put_ctx()) Process leaving 00000100:00000010:2.0:1773941878.068641:0:34772:0:(client.c:598:ptlrpc_request_cache_free()) slab-freed '(req)': 888 at 00000000e87286be. 00000100:00000001:2.0:1773941878.068643:0:34772:0:(client.c:2824:__ptlrpc_free_req()) Process leaving 00000100:00000001:2.0:1773941878.068643:0:34772:0:(client.c:2883:__ptlrpc_req_put()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1773941878.068644:0:34772:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=1 : 1 : 1) 00000100:00000001:2.0:1773941878.068647:0:34772:0:(client.c:2614:ptlrpc_set_next_timeout()) Process entered 00000100:00000001:2.0:1773941878.068648:0:34772:0:(client.c:2642:ptlrpc_set_next_timeout()) Process leaving (rc=0 : 0 : 0) 00000100:00000001:2.0:1773941878.068649:0:34772:0:(ptlrpcd.c:252:ptlrpcd_check()) Process entered 00000100:00000001:2.0:1773941878.068651:0:34772:0:(ptlrpcd.c:351:ptlrpcd_check()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1773941879.088424:0:25480:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941879.088433:0:25480:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1773941879.088481:0:36408:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1773941879.088485:0:36408:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1773941879.088487:0:36408:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00000001:3.0:1773941880.091627:0:37451:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) 00010000:00000001:2.0:1773941880.112411:0:42:0:(ldlm_pool.c:458:ldlm_cli_pool_recalc()) Process entered 00010000:00000001:2.0:1773941880.112417:0:42:0:(ldlm_pool.c:462:ldlm_cli_pool_recalc()) Process leaving (rc=0 : 0 : 0) 00000020:00000001:3.0:1773941880.112497:0:36409:0:(genops.c:1914:obd_stale_export_get()) Process entered 00000020:00000001:3.0:1773941880.112503:0:36409:0:(genops.c:1928:obd_stale_export_get()) Process leaving (rc=0 : 0 : 0) 00010000:00010000:3.0:1773941880.112507:0:36409:0:(ldlm_lockd.c:2797:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000001:0.0:1773941880.213817:0:37487:0:(debug.c:644:libcfs_debug_mark_buffer()) ************************************************** 00000001:02000400:0.0:1773941880.213821:0:37487:0:(debug.c:645:libcfs_debug_mark_buffer()) DEBUG MARKER: conf-sanity test_162: @@@@@@ FAIL: MDT start failed 00000001:00000001:0.0:1773941880.215754:0:37487:0:(debug.c:647:libcfs_debug_mark_buffer()) ************************************************** 10000000:00000001:2.0:1773941880.624435:0:36411:0:(mgc_request.c:117:config_log_put()) Process entered 10000000:00000001:2.0:1773941880.624441:0:36411:0:(mgc_request.c:120:config_log_put()) Process leaving 00000080:00000001:0.0:1773941880.711375:0:37585:0:(llite_lib.c:4092:ll_show_options()) Process leaving (rc=0 : 0 : 0) Debug log: 565 lines, 565 kept, 0 dropped, 0 bad.