ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность
openmpi/ompi/mca/btl/ud/btl_ud_endpoint.h

102 строки
3.2 KiB
C
Исходник Обычный вид История

Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2005 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2006 Sandia National Laboratories. All rights
* reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#ifndef MCA_BTL_IB_ENDPOINT_H
#define MCA_BTL_IB_ENDPOINT_H
#include "opal/class/opal_list.h"
#include "opal/event/event.h"
#include "ompi/mca/pml/pml.h"
#include "ompi/mca/btl/btl.h"
#include "btl_ud_frag.h"
#include "btl_ud.h"
#include <errno.h>
#include <string.h>
#include <infiniband/verbs.h>
#include "ompi/mca/btl/base/btl_base_error.h"
#include "ompi/mca/mpool/openib/mpool_openib.h"
#if defined(c_plusplus) || defined(__cplusplus)
extern "C" {
#endif
OBJ_CLASS_DECLARATION(mca_btl_ud_endpoint_t);
struct mca_btl_ud_frag_t;
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
struct mca_btl_ud_addr_t {
uint32_t qp_num_hp;
uint32_t qp_num_lp;
/* QP number (Low and High priority) */
uint32_t psn_hp;
uint32_t psn_lp;
/* Port sequence number (Low and High) */
uint16_t lid;
uint16_t subnet;
/* Local Identifier & Subnet */
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
};
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
typedef struct mca_btl_ud_addr_t mca_btl_ud_addr_t;
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
/**
* An abstraction that represents a connection to a endpoint process.
* An instance of mca_btl_base_endpoint_t is associated w/ each process
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
* and BTL pair and address information is exchanged at startup.
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
* The UD BTL is connectionless, so no connection is ever established.
*/
struct mca_btl_base_endpoint_t {
opal_list_item_t super;
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
/*opal_mutex_t endpoint_lock;*/
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
/**< lock for concurrent access to endpoint state */
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
mca_btl_ud_addr_t rem_addr;
/**< Remote address information */
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
struct ibv_ah* rmt_ah_hp;
struct ibv_ah* rmt_ah_lp;
Change over from lazy connection setup to setting up at initialization time. UD is connectionless, and as long as peers are statically assigned to QPs, there is no reason to set up the adressing information lazily. Lots of code was axed, as endpoints no longer have state. Removed a number of other elements in the endpoint struct to make it as lightweight as possible. I was able to remove an entire function call/branch in the send path, which I believe is the main contributor to a 2us drop in NetPIPE latency. Some whitespace cleanups as well. Passes IBM test suite, and all but certain intel tests that were failing before the change, over ob1 PML. This commit was SVN r10494.
2006-06-23 16:50:50 +00:00
/**< Remote Address Handle (Low and High) */
Finally committing the UD BTL. UD is the Unreliable Datagram transport for Infiniband, specifically OpenIB. This BTL is derived from the existing openib BTL, which is RC (Reliable Connection) based. Still a work in progress, as there is a lot of work left to do. Specifically, performance, scalability, and flow control need to be addressed. Currently I'm playing around with different methods for handling receive buffers, as well as profiling to figure out where the time is going. This commit was SVN r10271.
2006-06-09 00:13:45 +00:00
};
typedef struct mca_btl_base_endpoint_t mca_btl_base_endpoint_t;
typedef mca_btl_base_endpoint_t mca_btl_ud_endpoint_t;
inline int mca_btl_ud_endpoint_post_send(struct mca_btl_ud_module_t* ud_btl,
mca_btl_ud_endpoint_t * endpoint,
struct mca_btl_ud_frag_t * frag);
int mca_btl_ud_endpoint_init_qp(
mca_btl_base_module_t* btl,
struct ibv_cq* cq,
#ifdef OMPI_MCA_BTL_OPENIB_HAVE_SRQ
struct ibv_srq* srq,
#endif
struct ibv_qp** qp,
uint32_t lcl_psn);
#if defined(c_plusplus) || defined(__cplusplus)
}
#endif
#endif
Копировать постоянную ссылку