2005-08-02 17:20:50 +04:00
|
|
|
/*
|
2005-11-05 22:57:48 +03:00
|
|
|
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
|
|
|
|
* University Research and Technology
|
|
|
|
* Corporation. All rights reserved.
|
2006-08-24 20:38:08 +04:00
|
|
|
* Copyright (c) 2004-2006 The University of Tennessee and The University
|
2005-11-05 22:57:48 +03:00
|
|
|
* of Tennessee Research Foundation. All rights
|
|
|
|
* reserved.
|
2015-06-24 06:59:57 +03:00
|
|
|
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
2005-08-02 17:20:50 +04:00
|
|
|
* University of Stuttgart. All rights reserved.
|
|
|
|
* Copyright (c) 2004-2005 The Regents of the University of California.
|
|
|
|
* All rights reserved.
|
2010-04-14 18:18:35 +04:00
|
|
|
* Copyright (c) 2010 Oracle and/or its affiliates. All rights reserved
|
2005-08-02 17:20:50 +04:00
|
|
|
* $COPYRIGHT$
|
2015-06-24 06:59:57 +03:00
|
|
|
*
|
2005-08-02 17:20:50 +04:00
|
|
|
* Additional copyrights may follow
|
2015-06-24 06:59:57 +03:00
|
|
|
*
|
2005-08-02 17:20:50 +04:00
|
|
|
* $HEADER$
|
|
|
|
*/
|
|
|
|
|
|
|
|
#ifndef MCA_BTL_TCP_PROC_H
|
|
|
|
#define MCA_BTL_TCP_PROC_H
|
|
|
|
|
|
|
|
#include "opal/class/opal_object.h"
|
George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
2014-07-26 04:47:28 +04:00
|
|
|
#include "opal/util/proc.h"
|
2005-08-02 17:20:50 +04:00
|
|
|
#include "btl_tcp.h"
|
|
|
|
#include "btl_tcp_addr.h"
|
|
|
|
#include "btl_tcp_endpoint.h"
|
|
|
|
|
2009-08-20 15:42:18 +04:00
|
|
|
BEGIN_C_DECLS
|
2005-08-02 17:20:50 +04:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Represents the state of a remote process and the set of addresses
|
|
|
|
* that it exports. Also cache an instance of mca_btl_base_endpoint_t for
|
|
|
|
* each
|
|
|
|
* BTL instance that attempts to open a connection to the process.
|
|
|
|
*/
|
|
|
|
struct mca_btl_tcp_proc_t {
|
George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
2014-07-26 04:47:28 +04:00
|
|
|
opal_list_item_t super;
|
2005-08-02 17:20:50 +04:00
|
|
|
/**< allow proc to be placed on a list */
|
|
|
|
|
2014-11-06 09:39:25 +03:00
|
|
|
opal_proc_t *proc_opal;
|
George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
2014-07-26 04:47:28 +04:00
|
|
|
/**< pointer to corresponding opal_proc_t */
|
2005-08-02 17:20:50 +04:00
|
|
|
|
|
|
|
struct mca_btl_tcp_addr_t* proc_addrs;
|
|
|
|
/**< array of addresses exported by peer */
|
|
|
|
|
2015-06-24 06:59:57 +03:00
|
|
|
size_t proc_addr_count;
|
2005-08-02 17:20:50 +04:00
|
|
|
/**< number of addresses published by endpoint */
|
|
|
|
|
2015-06-24 06:59:57 +03:00
|
|
|
struct mca_btl_base_endpoint_t **proc_endpoints;
|
|
|
|
/**< array of endpoints that have been created to access this proc */
|
2005-08-02 17:20:50 +04:00
|
|
|
|
2015-06-24 06:59:57 +03:00
|
|
|
size_t proc_endpoint_count;
|
2005-08-02 17:20:50 +04:00
|
|
|
/**< number of endpoints */
|
|
|
|
|
2015-06-24 06:59:57 +03:00
|
|
|
opal_mutex_t proc_lock;
|
2005-08-02 17:20:50 +04:00
|
|
|
/**< lock to protect against concurrent access to proc state */
|
|
|
|
};
|
|
|
|
typedef struct mca_btl_tcp_proc_t mca_btl_tcp_proc_t;
|
2006-08-24 20:38:08 +04:00
|
|
|
OBJ_CLASS_DECLARATION(mca_btl_tcp_proc_t);
|
2005-08-02 17:20:50 +04:00
|
|
|
|
2008-02-14 00:53:00 +03:00
|
|
|
/* the highest possible interface kernel index we can handle */
|
|
|
|
#define MAX_KERNEL_INTERFACE_INDEX 65536
|
|
|
|
|
|
|
|
/* the maximum number of kernel interfaces we can handle */
|
|
|
|
#define MAX_KERNEL_INTERFACES 8
|
|
|
|
|
2010-04-14 18:18:35 +04:00
|
|
|
/* The maximum number of interfaces that we can have and use the
|
|
|
|
* recursion code for determining the best set of connections. When
|
|
|
|
* the number is greater than this, we switch to a simpler algorithm
|
|
|
|
* to speed things up. */
|
|
|
|
#define MAX_PERMUTATION_INTERFACES 8
|
|
|
|
|
2008-02-14 00:53:00 +03:00
|
|
|
/*
|
|
|
|
* FIXME: this should probably be part of an ompi list, so we need the
|
|
|
|
* appropriate definitions
|
|
|
|
*/
|
|
|
|
|
|
|
|
struct mca_btl_tcp_interface_t {
|
|
|
|
struct sockaddr_storage* ipv4_address;
|
|
|
|
struct sockaddr_storage* ipv6_address;
|
|
|
|
mca_btl_tcp_addr_t* ipv4_endpoint_addr;
|
|
|
|
mca_btl_tcp_addr_t* ipv6_endpoint_addr;
|
|
|
|
uint32_t ipv4_netmask;
|
|
|
|
uint32_t ipv6_netmask;
|
|
|
|
int kernel_index;
|
|
|
|
int peer_interface;
|
|
|
|
int index;
|
|
|
|
int inuse;
|
|
|
|
};
|
|
|
|
|
|
|
|
typedef struct mca_btl_tcp_interface_t mca_btl_tcp_interface_t;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* describes the quality of a possible connection between a local and
|
|
|
|
* a remote network interface
|
|
|
|
*/
|
2015-06-24 06:59:57 +03:00
|
|
|
enum mca_btl_tcp_connection_quality {
|
2008-02-14 00:53:00 +03:00
|
|
|
CQ_NO_CONNECTION,
|
|
|
|
CQ_PRIVATE_DIFFERENT_NETWORK,
|
|
|
|
CQ_PRIVATE_SAME_NETWORK,
|
|
|
|
CQ_PUBLIC_DIFFERENT_NETWORK,
|
|
|
|
CQ_PUBLIC_SAME_NETWORK
|
|
|
|
};
|
|
|
|
|
|
|
|
|
2014-11-06 09:39:25 +03:00
|
|
|
mca_btl_tcp_proc_t* mca_btl_tcp_proc_create(opal_proc_t* proc);
|
George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
2014-07-26 04:47:28 +04:00
|
|
|
mca_btl_tcp_proc_t* mca_btl_tcp_proc_lookup(const opal_process_name_t* name);
|
2005-08-02 17:20:50 +04:00
|
|
|
int mca_btl_tcp_proc_insert(mca_btl_tcp_proc_t*, mca_btl_base_endpoint_t*);
|
|
|
|
int mca_btl_tcp_proc_remove(mca_btl_tcp_proc_t*, mca_btl_base_endpoint_t*);
|
2014-12-16 04:27:32 +03:00
|
|
|
void mca_btl_tcp_proc_accept(mca_btl_tcp_proc_t*, struct sockaddr*, int);
|
2007-04-25 05:55:40 +04:00
|
|
|
bool mca_btl_tcp_proc_tosocks(mca_btl_tcp_addr_t*, struct sockaddr_storage*);
|
2005-08-02 17:20:50 +04:00
|
|
|
|
|
|
|
/**
|
|
|
|
* Inlined function to return local TCP proc instance.
|
|
|
|
*/
|
|
|
|
|
|
|
|
static inline mca_btl_tcp_proc_t* mca_btl_tcp_proc_local(void)
|
|
|
|
{
|
|
|
|
if(NULL == mca_btl_tcp_component.tcp_local)
|
George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
2014-07-26 04:47:28 +04:00
|
|
|
mca_btl_tcp_component.tcp_local = mca_btl_tcp_proc_create(opal_proc_local_get());
|
2005-08-02 17:20:50 +04:00
|
|
|
return mca_btl_tcp_component.tcp_local;
|
|
|
|
}
|
|
|
|
|
2009-08-20 15:42:18 +04:00
|
|
|
END_C_DECLS
|
2005-08-02 17:20:50 +04:00
|
|
|
#endif
|