ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность

As per the RFC, bring in the ORTE async progress code and the rewrite of OOB:

Просмотр исходного кода 
*** THIS RFC INCLUDES A MINOR CHANGE TO THE MPI-RTE INTERFACE ***

Note: during the course of this work, it was necessary to completely separate the MPI and RTE progress engines. There were multiple places in the MPI layer where ORTE_WAIT_FOR_COMPLETION was being used. A new OMPI_WAIT_FOR_COMPLETION macro was created (defined in ompi/mca/rte/rte.h) that simply cycles across opal_progress until the provided flag becomes false. Places where the MPI layer blocked waiting for RTE to complete an event have been modified to use this macro.

***************************************************************************************

I am reissuing this RFC because of the time that has passed since its original release. Since its initial release and review, I have debugged it further to ensure it fully supports tests like loop_spawn. It therefore seems ready for merge back to the trunk. Given its prior review, I have set the timeout for one week.

The code is in  https://bitbucket.org/rhc/ompi-oob2


WHAT:    Rewrite of ORTE OOB

WHY:       Support asynchronous progress and a host of other features

WHEN:    Wed, August 21

SYNOPSIS:
The current OOB has served us well, but a number of limitations have been identified over the years. Specifically:

* it is only progressed when called via opal_progress, which can lead to hangs or recursive calls into libevent (which is not supported by that code)

* we've had issues when multiple NICs are available as the code doesn't "shift" messages between transports - thus, all nodes had to be available via the same TCP interface.

* the OOB "unloads" incoming opal_buffer_t objects during the transmission, thus preventing use of OBJ_RETAIN in the code when repeatedly sending the same message to multiple recipients

* there is no failover mechanism across NICs - if the selected NIC (or its attached switch) fails, we are forced to abort

* only one transport (i.e., component) can be "active"


The revised OOB resolves these problems:

* async progress is used for all application processes, with the progress thread blocking in the event library

* each available TCP NIC is supported by its own TCP module. The ability to asynchronously progress each module independently is provided, but not enabled by default (a runtime MCA parameter turns it "on")

* multi-address TCP NICs (e.g., a NIC with both an IPv4 and IPv6 address, or with virtual interfaces) are supported - reachability is determined by comparing the contact info for a peer against all addresses within the range covered by the address/mask pairs for the NIC.

* a message that arrives on one TCP NIC is automatically shifted to whatever NIC that is connected to the next "hop" if that peer cannot be reached by the incoming NIC. If no TCP module will reach the peer, then the OOB attempts to send the message via all other available components - if none can reach the peer, then an "error" is reported back to the RML, which then calls the errmgr for instructions.

* opal_buffer_t now conforms to standard object rules re OBJ_RETAIN as we no longer "unload" the incoming object

* NIC failure is reported to the TCP component, which then tries to resend the message across any other available TCP NIC. If that doesn't work, then the message is given back to the OOB base to try using other components. If all that fails, then the error is reported to the RML, which reports to the errmgr for instructions

* obviously from the above, multiple OOB components (e.g., TCP and UD) can be active in parallel

* the matching code has been moved to the RML (and out of the OOB/TCP component) so it is independent of transport

* routing is done by the individual OOB modules (as opposed to the RML). Thus, both routed and non-routed transports can simultaneously be active

* all blocking send/recv APIs have been removed. Everything operates asynchronously.


KNOWN LIMITATIONS:

* although provision is made for component failover as described above, the code for doing so has not been fully implemented yet. At the moment, if all connections for a given peer fail, the errmgr is notified of a "lost connection", which by default results in termination of the job if it was a lifeline

* the IPv6 code is present and compiles, but is not complete. Since the current IPv6 support in the OOB doesn't work anyway, I don't consider this a blocker

* routing is performed at the individual module level, yet the active routed component is selected on a global basis. We probably should update that to reflect that different transports may need/choose to route in different ways

* obviously, not every error path has been tested nor necessarily covered

* determining abnormal termination is more challenging than in the old code as we now potentially have multiple ways of connecting to a process. Ideally, we would declare "connection failed" when *all* transports can no longer reach the process, but that requires some additional (possibly complex) code. For now, the code replicates the old behavior only somewhat modified - i.e., if a module sees its connection fail, it checks to see if it is a lifeline. If so, it notifies the errmgr that the lifeline is lost - otherwise, it notifies the errmgr that a non-lifeline connection was lost.

* reachability is determined solely on the basis of a shared subnet address/mask - more sophisticated algorithms (e.g., the one used in the tcp btl) are required to handle routing via gateways

* the RML needs to assign sequence numbers to each message on a per-peer basis. The receiving RML will then deliver messages in order, thus preventing out-of-order messaging in the case where messages travel across different transports or a message needs to be redirected/resent due to failure of a NIC

This commit was SVN r29058.
Этот коммит содержится в:
Ralph Castain 2013-08-22 16:37:40 +00:00
родитель 63d10d2d0d
Коммит a200e4f865
187 изменённых файлов: 8647 добавлений и 8906 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

17
config/orte_configure_options.m4
Просмотреть файл

@ -88,23 +88,6 @@ AC_DEFINE_UNQUOTED([ORTE_ENABLE_HEARTBEAT],
[$orte_want_heartbeats],
[Whether we want daemon heartbeat monitoring enabled])
#
# Do we want a separate orte progress thread?
AC_MSG_CHECKING([if want orte progress threads])
AC_ARG_ENABLE([orte-progress-threads],
[AC_HELP_STRING([--enable-orte-progress-threads],
[Enable orte progress thread - for experiment by developers only! (default: disabled)])])
if test "$enable_orte_progress_threads" = "yes"; then
AC_MSG_RESULT([yes])
orte_enable_progress_threads=1
else
AC_MSG_RESULT([no])
orte_enable_progress_threads=0
fi
AC_DEFINE_UNQUOTED([ORTE_ENABLE_PROGRESS_THREADS],
[$orte_enable_progress_threads],
[Whether we want orte progress threads enabled])
AC_MSG_CHECKING([if want orte static ports])
AC_ARG_ENABLE([orte-static-ports],
[AC_HELP_STRING([--enable-orte-static-ports],

4
contrib/build-mca-comps-outside-of-tree/btl_tcp2.addr.h
Просмотреть файл

@ -42,7 +42,7 @@ struct mca_btl_tcp2_addr_t {
machines (read: byte order), so use network byte order
for everything and don't add padding
*/
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct in6_addr addr_inet; /**< IPv4/IPv6 listen address > */
#else
/* Bug, FIXME: needs testing */
@ -65,7 +65,7 @@ struct mca_btl_tcp2_addr_t {
typedef struct mca_btl_tcp2_addr_t mca_btl_tcp2_addr_t;
#define MCA_BTL_TCP_AF_INET 0
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
# define MCA_BTL_TCP_AF_INET6 1
#endif

2
contrib/build-mca-comps-outside-of-tree/btl_tcp2.h
Просмотреть файл

@ -71,7 +71,7 @@ struct mca_btl_tcp2_component_t {
unsigned short tcp_listen_port; /**< IPv4 listen port */
int32_t tcp_port_min; /**< IPv4 minimum port */
int32_t tcp_port_range; /**< IPv4 port range */
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
opal_event_t tcp6_recv_event; /**< recv event for IPv6 listen socket */
int tcp6_listen_sd; /**< IPv6 listen socket for incoming connection requests */
unsigned short tcp6_listen_port; /**< IPv6 listen port */

4
contrib/build-mca-comps-outside-of-tree/btl_tcp2_addr.h
Просмотреть файл

@ -42,7 +42,7 @@ struct mca_btl_tcp2_addr_t {
machines (read: byte order), so use network byte order
for everything and don't add padding
*/
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct in6_addr addr_inet; /**< IPv4/IPv6 listen address > */
#else
/* Bug, FIXME: needs testing */
@ -65,7 +65,7 @@ struct mca_btl_tcp2_addr_t {
typedef struct mca_btl_tcp2_addr_t mca_btl_tcp2_addr_t;
#define MCA_BTL_TCP_AF_INET 0
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
# define MCA_BTL_TCP_AF_INET6 1
#endif

32
contrib/build-mca-comps-outside-of-tree/btl_tcp2_component.c
Просмотреть файл

@ -40,7 +40,7 @@
#ifdef HAVE_ARPA_INET_H
#include <arpa/inet.h>
#endif
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
# ifdef HAVE_NETDB_H
# include <netdb.h>
# endif
@ -175,7 +175,7 @@ int mca_btl_tcp2_component_open(void)
/* initialize state */
mca_btl_tcp2_component.tcp_listen_sd = -1;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
mca_btl_tcp2_component.tcp6_listen_sd = -1;
#endif
mca_btl_tcp2_component.tcp_num_btls=0;
@ -234,7 +234,7 @@ int mca_btl_tcp2_component_open(void)
mca_btl_tcp2_param_register_int( "port_range_v4", message,
(0x1 << 16) - mca_btl_tcp2_component.tcp_port_min - 1);
free(message);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
mca_btl_tcp2_component.tcp6_port_min =
mca_btl_tcp2_param_register_int( "port_min_v6",
"The minimum port where the TCP BTL will try to bind (default 1024)", 1024 );
@ -304,7 +304,7 @@ int mca_btl_tcp2_component_close(void)
CLOSE_THE_SOCKET(mca_btl_tcp2_component.tcp_listen_sd);
mca_btl_tcp2_component.tcp_listen_sd = -1;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (mca_btl_tcp2_component.tcp6_listen_sd >= 0) {
opal_event_del(&mca_btl_tcp2_component.tcp6_recv_event);
CLOSE_THE_SOCKET(mca_btl_tcp2_component.tcp6_listen_sd);
@ -642,7 +642,7 @@ static int mca_btl_tcp2_component_create_listen(uint16_t af_family)
mca_btl_tcp2_set_socket_options(sd);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
struct addrinfo hints, *res = NULL;
int error;
@ -698,19 +698,19 @@ static int mca_btl_tcp2_component_create_listen(uint16_t af_family)
range = mca_btl_tcp2_component.tcp_port_range;
port = mca_btl_tcp2_component.tcp_port_min;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
range = mca_btl_tcp2_component.tcp6_port_range;
port = mca_btl_tcp2_component.tcp6_port_min;
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
for( index = 0; index < range; index++ ) {
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
((struct sockaddr_in6*) &inaddr)->sin6_port = htons(port + index);
#else
((struct sockaddr_in*) &inaddr)->sin_port = htons(port + index);
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
if(bind(sd, (struct sockaddr*)&inaddr, addrlen) < 0) {
if( (EADDRINUSE == opal_socket_errno) || (EADDRNOTAVAIL == opal_socket_errno) ) {
continue;
@ -727,13 +727,13 @@ static int mca_btl_tcp2_component_create_listen(uint16_t af_family)
mca_btl_tcp2_component.tcp_port_min,
mca_btl_tcp2_component.tcp_port_min + range));
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
BTL_ERROR(("bind6() failed: no port available in the range [%d..%d]",
mca_btl_tcp2_component.tcp6_port_min,
mca_btl_tcp2_component.tcp6_port_min + range));
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
CLOSE_THE_SOCKET(sd);
return OMPI_ERROR;
}
@ -750,7 +750,7 @@ static int mca_btl_tcp2_component_create_listen(uint16_t af_family)
mca_btl_tcp2_component.tcp_listen_port = ((struct sockaddr_in*) &inaddr)->sin_port;
mca_btl_tcp2_component.tcp_listen_sd = sd;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
mca_btl_tcp2_component.tcp6_listen_port = ((struct sockaddr_in6*) &inaddr)->sin6_port;
mca_btl_tcp2_component.tcp6_listen_sd = sd;
@ -790,7 +790,7 @@ static int mca_btl_tcp2_component_create_listen(uint16_t af_family)
0 );
opal_event_add(&mca_btl_tcp2_component.tcp_recv_event, 0);
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
opal_event_set(opal_event_base, &mca_btl_tcp2_component.tcp6_recv_event,
mca_btl_tcp2_component.tcp6_listen_sd,
@ -865,7 +865,7 @@ static int mca_btl_tcp2_component_exchange(void)
opal_ifindextokindex (index);
current_addr++;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if ((AF_INET6 == my_ss.ss_family) &&
(6 != mca_btl_tcp2_component.tcp_disable_family)) {
memcpy(&addrs[current_addr].addr_inet,
@ -947,7 +947,7 @@ mca_btl_base_module_t** mca_btl_tcp2_component_init(int *num_btl_modules,
if(OMPI_SUCCESS != (ret = mca_btl_tcp2_component_create_listen(AF_INET) )) {
return 0;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if((ret = mca_btl_tcp2_component_create_listen(AF_INET6)) != OMPI_SUCCESS) {
if (!(OMPI_ERR_IN_ERRNO == OPAL_SOS_GET_ERROR_CODE(ret) &&
EAFNOSUPPORT == opal_socket_errno)) {
@ -993,7 +993,7 @@ static void mca_btl_tcp2_component_accept_handler( int incoming_sd,
void* unused )
{
while(true) {
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct sockaddr_in6 addr;
#else
struct sockaddr_in addr;

8
contrib/build-mca-comps-outside-of-tree/btl_tcp2_endpoint.c
Просмотреть файл

@ -129,7 +129,7 @@ static void mca_btl_tcp2_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, co
char src[64];
char dst[64];
int sndbuf,rcvbuf,nodelay,flags;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct sockaddr_storage inaddr;
#else
struct sockaddr_in inaddr;
@ -138,7 +138,7 @@ static void mca_btl_tcp2_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, co
opal_socklen_t addrlen = sizeof(inaddr);
getsockname(btl_endpoint->endpoint_sd, (struct sockaddr*)&inaddr, &addrlen);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
char *address;
address = (char *) opal_net_get_hostname((struct sockaddr*) &inaddr);
@ -150,7 +150,7 @@ static void mca_btl_tcp2_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, co
sprintf(src, "%s", inet_ntoa(inaddr.sin_addr));
#endif
getpeername(btl_endpoint->endpoint_sd, (struct sockaddr*)&inaddr, &addrlen);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
char *address;
address = (char *) opal_net_get_hostname ((struct sockaddr*) &inaddr);
@ -536,7 +536,7 @@ static int mca_btl_tcp2_endpoint_start_connect(mca_btl_base_endpoint_t* btl_endp
uint16_t af_family = AF_INET;
opal_socklen_t addrlen = sizeof(struct sockaddr_in);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == btl_endpoint->endpoint_addr->addr_family) {
af_family = AF_INET6;
addrlen = sizeof (struct sockaddr_in6);

6
contrib/build-mca-comps-outside-of-tree/btl_tcp2_proc.c
Просмотреть файл

@ -158,7 +158,7 @@ mca_btl_tcp2_proc_t* mca_btl_tcp2_proc_create(ompi_proc_t* ompi_proc)
if (MCA_BTL_TCP_AF_INET == btl_proc->proc_addrs[i].addr_family) {
btl_proc->proc_addrs[i].addr_family = AF_INET;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (MCA_BTL_TCP_AF_INET6 == btl_proc->proc_addrs[i].addr_family) {
btl_proc->proc_addrs[i].addr_family = AF_INET6;
}
@ -740,7 +740,7 @@ bool mca_btl_tcp2_proc_accept(mca_btl_tcp2_proc_t* btl_proc, struct sockaddr* ad
continue;
}
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
if( memcmp( &btl_endpoint->endpoint_addr->addr_inet,
&(((struct sockaddr_in6*)addr)->sin6_addr),
@ -777,7 +777,7 @@ bool mca_btl_tcp2_proc_tosocks(mca_btl_tcp2_addr_t* proc_addr,
&proc_addr->addr_inet, sizeof(struct in_addr));
((struct sockaddr_in*)output)->sin_port = proc_addr->addr_port;
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
{
struct sockaddr_in6* inaddr = (struct sockaddr_in6*)output;

77
contrib/search_compare.pl
Просмотреть файл

@ -10,6 +10,7 @@
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2013 Intel Inc. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
@ -20,6 +21,8 @@
use File::Find;
use File::Basename;
use File::Compare;
use Text::Diff;
use Getopt::Long;
if (scalar(@ARGV) != 2) {
print "Usage: search_compare.pl src_dir target_dir\n";
@ -34,6 +37,25 @@ my @src_tree = ();
my @tgt_tree = ();
my $flag;
my $ok = Getopt::Long::GetOptions("diff" => \$diff_arg,
"debug|d" => \$debug_arg,
"help|h" => \$help_arg,
"ignore=s" => \$exclude_arg,
);
if (!$ok || $help_arg) {
print "Invalid command line argument.\n\n"
if (!$ok);
print "Options:
--diff | -diff Output a diff of files that were modified - turns
off the output of files added/deleted
--debug | -d Output lots of debug information
--help | -h This help list
--ignore | -ignore Name of file containing typical ignore list - e.g.,
.hgignore or .gitignore\n";
my_exit($ok ? 0 : 1);
}
sub construct {
# don't process directories or links, and dont' recurse down
# "special" directories
@ -84,6 +106,7 @@ my $tgt_file;
my @modified = ();
my @src_pared = ();
my $i;
my $d;
foreach $src (@src_tree) {
# strip the leading elements of the path that was given to us
$src_file = substr($src, $len_src_dir);
@ -97,42 +120,46 @@ foreach $src (@src_tree) {
# file has been found - ignore it
$found = 1;
if (compare($src, $tgt) != 0) {
push(@modified, $src);
$d = diff $src, $tgt;
print "Index: $tgt\n";
print "===================================================================\n";
print "$d\n";
# push(@modified, $src);
}
# remove this file from the target tree as it has been found
# splice @tgt_tree, $i, 1;
break;
}
}
if ($found == 0) {
print "Add: " . $src . "\n";
} else {
push(@src_pared, $src);
}
# if ($found == 0) {
# print "Add: " . $src . "\n";
## } else {
# push(@src_pared, $src);
# }
}
print "\n";
# print a list of files in the target tree that need to be deleted
foreach $tgt (@tgt_tree) {
$found = 0;
$tgt_file = substr($tgt, $len_tgt_dir);
foreach $src (@src_pared) {
$src_file = substr($src, $len_src_dir);
if ($src_file eq $tgt_file) {
# file has been found - ignore it
$found = 1;
break;
}
}
if ($found == 0) {
print "Delete: " . $tgt . "\n";
}
}
#foreach $tgt (@tgt_tree) {
# $found = 0;
# $tgt_file = substr($tgt, $len_tgt_dir);
# foreach $src (@src_pared) {
# $src_file = substr($src, $len_src_dir);
# if ($src_file eq $tgt_file) {
# # file has been found - ignore it
# $found = 1;
# break;
# }
# }
# if ($found == 0) {
# print "Delete: " . $tgt . "\n";
# }
#}
print "\n";
#print "\n";
# print a list of files that have been modified
foreach $tgt (@modified) {
print "Modified: " . $tgt . "\n";
}
#foreach $tgt (@modified) {
# print "Modified: " . $tgt . "\n";
#}

54
ompi/communicator/comm_cid.c
Просмотреть файл

@ -14,9 +14,10 @@
* Copyright (c) 2007 Voltaire All rights reserved.
* Copyright (c) 2006-2010 University of Houston. All rights reserved.
* Copyright (c) 2009 Sun Microsystems, Inc. All rights reserved.
* Copyright (c) 2012 Los Alamos National Security, LLC. All rights
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2012 Oak Ridge National Labs. All rights reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -762,6 +763,23 @@ static int ompi_comm_allreduce_intra_bridge (int *inbuf, int *outbuf,
return (rc);
}
typedef struct {
opal_buffer_t buf;
bool active;
} comm_cid_return_t;
static void comm_cid_recv(int status,
ompi_process_name_t* peer,
opal_buffer_t* buffer,
ompi_rml_tag_t tag,
void* cbdata)
{
comm_cid_return_t *rcid = (comm_cid_return_t*)cbdata;
opal_dss.copy_payload(&rcid->buf, buffer);
rcid->active = false;
}
/* Arguments not used in this implementation:
* - bridgecomm
*
@ -782,6 +800,7 @@ static int ompi_comm_allreduce_intra_oob (int *inbuf, int *outbuf,
int local_leader, local_rank;
ompi_process_name_t *remote_leader=NULL;
int32_t size_count;
comm_cid_return_t rcid;
local_leader = (*((int*)lleader));
remote_leader = (ompi_process_name_t*)rleader;
@ -809,37 +828,46 @@ static int ompi_comm_allreduce_intra_oob (int *inbuf, int *outbuf,
if (local_rank == local_leader ) {
opal_buffer_t *sbuf;
opal_buffer_t *rbuf;
sbuf = OBJ_NEW(opal_buffer_t);
rbuf = OBJ_NEW(opal_buffer_t);
if (OPAL_SUCCESS != (rc = opal_dss.pack(sbuf, tmpbuf, (int32_t)count, OPAL_INT))) {
goto exit;
}
if ( send_first ) {
if (0 > (rc = ompi_rte_send_buffer(remote_leader, sbuf, OMPI_RML_TAG_COMM_CID_INTRA, 0))) {
if (0 > (rc = ompi_rte_send_buffer_nb(remote_leader, sbuf,
OMPI_RML_TAG_COMM_CID_INTRA,
ompi_rte_send_cbfunc, NULL))) {
goto exit;
}
if (0 > (rc = ompi_rte_recv_buffer(remote_leader, rbuf, OMPI_RML_TAG_COMM_CID_INTRA, 0))) {
goto exit;
OBJ_CONSTRUCT(&rcid.buf, opal_buffer_t);
rcid.active = true;
ompi_rte_recv_buffer_nb(remote_leader, OMPI_RML_TAG_COMM_CID_INTRA,
OMPI_RML_NON_PERSISTENT, comm_cid_recv, &rcid);
while (rcid.active) {
opal_progress();
}
}
else {
if (0 > (rc = ompi_rte_recv_buffer(remote_leader, rbuf, OMPI_RML_TAG_COMM_CID_INTRA, 0))) {
goto exit;
OBJ_CONSTRUCT(&rcid.buf, opal_buffer_t);
rcid.active = true;
ompi_rte_recv_buffer_nb(remote_leader, OMPI_RML_TAG_COMM_CID_INTRA,
OMPI_RML_NON_PERSISTENT, comm_cid_recv, &rcid);
while (rcid.active) {
opal_progress();
}
if (0 > (rc = ompi_rte_send_buffer(remote_leader, sbuf, OMPI_RML_TAG_COMM_CID_INTRA, 0))) {
if (0 > (rc = ompi_rte_send_buffer_nb(remote_leader, sbuf,
OMPI_RML_TAG_COMM_CID_INTRA,
ompi_rte_send_cbfunc, NULL))) {
goto exit;
}
}
if (OPAL_SUCCESS != (rc = opal_dss.unpack(rbuf, outbuf, &size_count, OPAL_INT))) {
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&rcid.buf, outbuf, &size_count, OPAL_INT))) {
goto exit;
}
OBJ_RELEASE(sbuf);
OBJ_RELEASE(rbuf);
OBJ_DESTRUCT(&rcid.buf);
count = (int)size_count;
if ( &ompi_mpi_op_max.op == op ) {

20
ompi/mca/btl/openib/connect/btl_openib_connect_oob.c
Просмотреть файл

@ -121,8 +121,6 @@ static void oob_component_register(void)
static int oob_component_query(mca_btl_openib_module_t *btl,
ompi_btl_openib_connect_base_module_t **cpc)
{
int rc;
/* If we have the transport_type member, check to ensure we're on
IB (this CPC will not work with iWarp). If we do not have the
transport_type member, then we must be < OFED v1.2, and
@ -148,17 +146,11 @@ static int oob_component_query(mca_btl_openib_module_t *btl,
ensure to only post it *once*, because another btl may have
come in before this and already posted it. */
if (!rml_recv_posted) {
rc = ompi_rte_recv_buffer_nb(OMPI_NAME_WILDCARD,
OMPI_RML_TAG_OPENIB,
OMPI_RML_PERSISTENT,
rml_recv_cb,
NULL);
if (OMPI_SUCCESS != rc) {
opal_output_verbose(5, ompi_btl_base_framework.framework_output,
"openib BTL: oob CPC system error %d (%s)",
rc, opal_strerror(rc));
return rc;
}
ompi_rte_recv_buffer_nb(OMPI_NAME_WILDCARD,
OMPI_RML_TAG_OPENIB,
OMPI_RML_PERSISTENT,
rml_recv_cb,
NULL);
rml_recv_posted = true;
}
@ -625,7 +617,7 @@ static int send_connect_data(mca_btl_base_endpoint_t* endpoint,
/* send to remote endpoint */
rc = ompi_rte_send_buffer_nb(&endpoint->endpoint_proc->proc_ompi->proc_name,
buffer, OMPI_RML_TAG_OPENIB, 0,
buffer, OMPI_RML_TAG_OPENIB,
rml_send_cb, NULL);
if (OMPI_SUCCESS != rc) {
OMPI_ERROR_LOG(rc);

2
ompi/mca/btl/tcp/btl_tcp.h
Просмотреть файл

@ -70,7 +70,7 @@ struct mca_btl_tcp_component_t {
unsigned short tcp_listen_port; /**< IPv4 listen port */
int tcp_port_min; /**< IPv4 minimum port */
int tcp_port_range; /**< IPv4 port range */
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
opal_event_t tcp6_recv_event; /**< recv event for IPv6 listen socket */
int tcp6_listen_sd; /**< IPv6 listen socket for incoming connection requests */
unsigned short tcp6_listen_port; /**< IPv6 listen port */

4
ompi/mca/btl/tcp/btl_tcp_addr.h
Просмотреть файл

@ -41,7 +41,7 @@ struct mca_btl_tcp_addr_t {
machines (read: byte order), so use network byte order
for everything and don't add padding
*/
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct in6_addr addr_inet; /**< IPv4/IPv6 listen address > */
#else
/* Bug, FIXME: needs testing */
@ -64,7 +64,7 @@ struct mca_btl_tcp_addr_t {
typedef struct mca_btl_tcp_addr_t mca_btl_tcp_addr_t;
#define MCA_BTL_TCP_AF_INET 0
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
# define MCA_BTL_TCP_AF_INET6 1
#endif

34
ompi/mca/btl/tcp/btl_tcp_component.c
Просмотреть файл

@ -42,7 +42,7 @@
#ifdef HAVE_ARPA_INET_H
#include <arpa/inet.h>
#endif
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
# ifdef HAVE_NETDB_H
# include <netdb.h>
# endif
@ -201,7 +201,7 @@ static int mca_btl_tcp_component_verify(void)
mca_btl_tcp_component.tcp_port_min );
mca_btl_tcp_component.tcp_port_min = 1024;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if( mca_btl_tcp_component.tcp6_port_min > USHRT_MAX ) {
opal_show_help("help-mpi-btl-tcp.txt", "invalid minimum port",
true, "v6", ompi_process_info.nodename,
@ -253,7 +253,7 @@ static int mca_btl_tcp_component_register(void)
(0x1 << 16) - mca_btl_tcp_component.tcp_port_min - 1,
OPAL_INFO_LVL_2, &mca_btl_tcp_component.tcp_port_range);
free(message);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
mca_btl_tcp_param_register_int( "port_min_v6",
"The minimum port where the TCP BTL will try to bind (default 1024)", 1024,
OPAL_INFO_LVL_2, & mca_btl_tcp_component.tcp6_port_min );
@ -350,7 +350,7 @@ static int mca_btl_tcp_component_open(void)
/* initialize state */
mca_btl_tcp_component.tcp_listen_sd = -1;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
mca_btl_tcp_component.tcp6_listen_sd = -1;
#endif
mca_btl_tcp_component.tcp_num_btls=0;
@ -405,7 +405,7 @@ static int mca_btl_tcp_component_close(void)
CLOSE_THE_SOCKET(mca_btl_tcp_component.tcp_listen_sd);
mca_btl_tcp_component.tcp_listen_sd = -1;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (mca_btl_tcp_component.tcp6_listen_sd >= 0) {
opal_event_del(&mca_btl_tcp_component.tcp6_recv_event);
CLOSE_THE_SOCKET(mca_btl_tcp_component.tcp6_listen_sd);
@ -754,7 +754,7 @@ static int mca_btl_tcp_component_create_listen(uint16_t af_family)
mca_btl_tcp_set_socket_options(sd);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
struct addrinfo hints, *res = NULL;
int error;
@ -810,19 +810,19 @@ static int mca_btl_tcp_component_create_listen(uint16_t af_family)
range = mca_btl_tcp_component.tcp_port_range;
port = mca_btl_tcp_component.tcp_port_min;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
range = mca_btl_tcp_component.tcp6_port_range;
port = mca_btl_tcp_component.tcp6_port_min;
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
for( index = 0; index < range; index++ ) {
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
((struct sockaddr_in6*) &inaddr)->sin6_port = htons(port + index);
#else
((struct sockaddr_in*) &inaddr)->sin_port = htons(port + index);
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
if(bind(sd, (struct sockaddr*)&inaddr, addrlen) < 0) {
if( (EADDRINUSE == opal_socket_errno) || (EADDRNOTAVAIL == opal_socket_errno) ) {
continue;
@ -839,13 +839,13 @@ static int mca_btl_tcp_component_create_listen(uint16_t af_family)
mca_btl_tcp_component.tcp_port_min,
mca_btl_tcp_component.tcp_port_min + range));
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
BTL_ERROR(("bind6() failed: no port available in the range [%d..%d]",
mca_btl_tcp_component.tcp6_port_min,
mca_btl_tcp_component.tcp6_port_min + range));
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
CLOSE_THE_SOCKET(sd);
return OMPI_ERROR;
}
@ -862,7 +862,7 @@ static int mca_btl_tcp_component_create_listen(uint16_t af_family)
mca_btl_tcp_component.tcp_listen_port = ((struct sockaddr_in*) &inaddr)->sin_port;
mca_btl_tcp_component.tcp_listen_sd = sd;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
mca_btl_tcp_component.tcp6_listen_port = ((struct sockaddr_in6*) &inaddr)->sin6_port;
mca_btl_tcp_component.tcp6_listen_sd = sd;
@ -902,7 +902,7 @@ static int mca_btl_tcp_component_create_listen(uint16_t af_family)
0 );
opal_event_add(&mca_btl_tcp_component.tcp_recv_event, 0);
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == af_family) {
opal_event_set(opal_event_base, &mca_btl_tcp_component.tcp6_recv_event,
mca_btl_tcp_component.tcp6_listen_sd,
@ -977,7 +977,7 @@ static int mca_btl_tcp_component_exchange(void)
opal_ifindextokindex (index);
current_addr++;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if ((AF_INET6 == my_ss.ss_family) &&
(6 != mca_btl_tcp_component.tcp_disable_family)) {
memcpy(&addrs[current_addr].addr_inet,
@ -1059,7 +1059,7 @@ mca_btl_base_module_t** mca_btl_tcp_component_init(int *num_btl_modules,
if(OMPI_SUCCESS != (ret = mca_btl_tcp_component_create_listen(AF_INET) )) {
return 0;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if((ret = mca_btl_tcp_component_create_listen(AF_INET6)) != OMPI_SUCCESS) {
if (!(OMPI_ERR_IN_ERRNO == ret &&
EAFNOSUPPORT == opal_socket_errno)) {
@ -1096,7 +1096,7 @@ static void mca_btl_tcp_component_accept_handler( int incoming_sd,
void* unused )
{
while(true) {
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct sockaddr_in6 addr;
#else
struct sockaddr_in addr;

8
ompi/mca/btl/tcp/btl_tcp_endpoint.c
Просмотреть файл

@ -131,7 +131,7 @@ static void mca_btl_tcp_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, con
char src[64];
char dst[64];
int sndbuf,rcvbuf,nodelay,flags;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct sockaddr_storage inaddr;
#else
struct sockaddr_in inaddr;
@ -140,7 +140,7 @@ static void mca_btl_tcp_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, con
opal_socklen_t addrlen = sizeof(inaddr);
getsockname(btl_endpoint->endpoint_sd, (struct sockaddr*)&inaddr, &addrlen);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
char *address;
address = (char *) opal_net_get_hostname((struct sockaddr*) &inaddr);
@ -152,7 +152,7 @@ static void mca_btl_tcp_endpoint_dump(mca_btl_base_endpoint_t* btl_endpoint, con
sprintf(src, "%s", inet_ntoa(inaddr.sin_addr));
#endif
getpeername(btl_endpoint->endpoint_sd, (struct sockaddr*)&inaddr, &addrlen);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
{
char *address;
address = (char *) opal_net_get_hostname ((struct sockaddr*) &inaddr);
@ -572,7 +572,7 @@ static int mca_btl_tcp_endpoint_start_connect(mca_btl_base_endpoint_t* btl_endpo
uint16_t af_family = AF_INET;
opal_socklen_t addrlen = sizeof(struct sockaddr_in);
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (AF_INET6 == btl_endpoint->endpoint_addr->addr_family) {
af_family = AF_INET6;
addrlen = sizeof (struct sockaddr_in6);

6
ompi/mca/btl/tcp/btl_tcp_proc.c
Просмотреть файл

@ -158,7 +158,7 @@ mca_btl_tcp_proc_t* mca_btl_tcp_proc_create(ompi_proc_t* ompi_proc)
if (MCA_BTL_TCP_AF_INET == btl_proc->proc_addrs[i].addr_family) {
btl_proc->proc_addrs[i].addr_family = AF_INET;
}
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
if (MCA_BTL_TCP_AF_INET6 == btl_proc->proc_addrs[i].addr_family) {
btl_proc->proc_addrs[i].addr_family = AF_INET6;
}
@ -745,7 +745,7 @@ bool mca_btl_tcp_proc_accept(mca_btl_tcp_proc_t* btl_proc, struct sockaddr* addr
continue;
}
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
if( memcmp( &btl_endpoint->endpoint_addr->addr_inet,
&(((struct sockaddr_in6*)addr)->sin6_addr),
@ -782,7 +782,7 @@ bool mca_btl_tcp_proc_tosocks(mca_btl_tcp_addr_t* proc_addr,
&proc_addr->addr_inet, sizeof(struct in_addr));
((struct sockaddr_in*)output)->sin_port = proc_addr->addr_port;
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
{
struct sockaddr_in6* inaddr = (struct sockaddr_in6*)output;

19
ompi/mca/common/ofacm/common_ofacm_oob.c
Просмотреть файл

@ -146,8 +146,6 @@ static void oob_component_register(void)
static int oob_component_query(ompi_common_ofacm_base_dev_desc_t *dev,
ompi_common_ofacm_base_module_t **cpc)
{
int rc;
if (oob_priority > 100) {
oob_priority = 100;
} else if (oob_priority < -1) {
@ -174,16 +172,11 @@ static int oob_component_query(ompi_common_ofacm_base_dev_desc_t *dev,
ensure to only post it *once*, because another btl may have
come in before this and already posted it. */
if (!rml_recv_posted) {
rc = ompi_rte_recv_buffer_nb(OMPI_NAME_WILDCARD,
OMPI_RML_TAG_OFACM,
OMPI_RML_PERSISTENT,
rml_recv_cb,
NULL);
if (OMPI_SUCCESS != rc) {
OFACM_VERBOSE(("OFACM: oob CPC system error %d (%s)",
rc, opal_strerror(rc)));
return rc;
}
ompi_rte_recv_buffer_nb(OMPI_NAME_WILDCARD,
OMPI_RML_TAG_OFACM,
OMPI_RML_PERSISTENT,
rml_recv_cb,
NULL);
rml_recv_posted = true;
}
@ -728,7 +721,7 @@ static int send_connect_data(ompi_common_ofacm_base_local_connection_context_t*
/* send to remote endpoint */
rc = ompi_rte_send_buffer_nb(&context->proc->proc_ompi->proc_name,
buffer, OMPI_RML_TAG_OFACM, 0,
buffer, OMPI_RML_TAG_OFACM,
rml_send_cb, NULL);
if (OMPI_SUCCESS != rc) {
OMPI_ERROR_LOG(rc);

50
ompi/mca/common/sm/common_sm_rml.c
Просмотреть файл

@ -11,7 +11,7 @@
* All rights reserved.
* Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved.
* Copyright (c) 2008-2010 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2010-2012 Los Alamos National Security, LLC.
* Copyright (c) 2010-2013 Los Alamos National Security, LLC.
* All rights reserved.
* $COPYRIGHT$
*
@ -36,6 +36,25 @@
#include <string.h>
#endif
typedef struct {
opal_buffer_t buf;
bool active;
int status;
} sm_return_t;
static void sml_recv(int status,
ompi_process_name_t* peer,
opal_buffer_t* buffer,
ompi_rml_tag_t tag,
void* cbdata)
{
sm_return_t *smr = (sm_return_t*)cbdata;
opal_dss.copy_payload(&smr->buf, buffer);
smr->active = false;
smr->status = status;
}
/* ////////////////////////////////////////////////////////////////////////// */
/**
* this routine assumes that sorted_procs is in the following state:
@ -53,10 +72,13 @@ mca_common_sm_rml_info_bcast(opal_shmem_ds_t *out_ds_buf,
int rc = OMPI_SUCCESS, tmprc;
char *msg_id_str_to_tx = NULL;
opal_buffer_t *buffer = NULL;
sm_return_t smr;
if (NULL == (buffer = OBJ_NEW(opal_buffer_t))) {
return OMPI_ERR_OUT_OF_RESOURCE;
}
OBJ_CONSTRUCT(&smr.buf, opal_buffer_t);
/* figure out if i am the root proc in the group. if i am, bcast the
* message the rest of the local procs. */
if (proc0) {
@ -83,8 +105,8 @@ mca_common_sm_rml_info_bcast(opal_shmem_ds_t *out_ds_buf,
/* first num_local_procs items should be local procs */
for (p = 1; p < num_local_procs; ++p) {
/* a potential future optimization: use non-blocking routines */
tmprc = ompi_rte_send_buffer(&(procs[p]->proc_name), buffer, tag,
0);
tmprc = ompi_rte_send_buffer_nb(&(procs[p]->proc_name), buffer, tag,
ompi_rte_send_cbfunc, NULL);
if (0 > tmprc) {
OMPI_ERROR_LOG(tmprc);
opal_progress_event_users_decrement();
@ -100,16 +122,24 @@ mca_common_sm_rml_info_bcast(opal_shmem_ds_t *out_ds_buf,
/* bump up the libevent polling frequency while we're in this RML recv,
* just to ensure we're checking libevent frequently. */
opal_progress_event_users_increment();
tmprc = ompi_rte_recv_buffer(&(procs[0]->proc_name), buffer, tag, 0);
smr.active = true;
smr.status = OMPI_ERROR;
ompi_rte_recv_buffer_nb(&(procs[0]->proc_name),tag,
OMPI_RML_NON_PERSISTENT,
sml_recv, &smr);
while (smr.active) {
opal_progress();
}
opal_progress_event_users_decrement();
if (0 > tmprc) {
OMPI_ERROR_LOG(tmprc);
rc = OMPI_ERROR;
if (OMPI_SUCCESS != smr.status) {
OMPI_ERROR_LOG(smr.status);
rc = smr.status;
goto out;
}
/* unpack the buffer */
num_vals = 1;
tmprc = opal_dss.unpack(buffer, &msg_id_str_to_tx, &num_vals,
tmprc = opal_dss.unpack(&smr.buf, &msg_id_str_to_tx, &num_vals,
OPAL_STRING);
if (0 > tmprc) {
OMPI_ERROR_LOG(OMPI_ERR_UNPACK_FAILURE);
@ -117,7 +147,7 @@ mca_common_sm_rml_info_bcast(opal_shmem_ds_t *out_ds_buf,
goto out;
}
num_vals = (int32_t)sizeof(opal_shmem_ds_t);
tmprc = opal_dss.unpack(buffer, out_ds_buf, &num_vals, OPAL_BYTE);
tmprc = opal_dss.unpack(&smr.buf, out_ds_buf, &num_vals, OPAL_BYTE);
if (0 > tmprc) {
OMPI_ERROR_LOG(OMPI_ERR_UNPACK_FAILURE);
rc = OMPI_ERROR;
@ -143,6 +173,6 @@ out:
free(msg_id_str_to_tx);
msg_id_str_to_tx = NULL;
}
OBJ_RELEASE(buffer);
OBJ_DESTRUCT(&smr.buf);
return rc;
}

8
ompi/mca/crcp/bkmrk/crcp_bkmrk_pml.c
Просмотреть файл

@ -3034,7 +3034,7 @@ ompi_crcp_base_pml_state_t* ompi_crcp_bkmrk_pml_ft_event(
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCPBR0);
ompi_rte_barrier(&coll);
ORTE_WAIT_FOR_COMPLETION(coll.active);
OMPI_WAIT_FOR_COMPLETION(coll.active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCP0);
@ -3103,7 +3103,7 @@ ompi_crcp_base_pml_state_t* ompi_crcp_bkmrk_pml_ft_event(
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_COREBR1);
ompi_rte_barrier(&coll);
ORTE_WAIT_FOR_COMPLETION(coll.active);
OMPI_WAIT_FOR_COMPLETION(coll.active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CORE2);
}
@ -6270,7 +6270,7 @@ static void display_all_timers(int state) {
}
else if( 2 == timing_enabled ) {
ompi_rte_barrier(&coll);
ORTE_WAIT_FOR_COMPLETION(coll.active);
OMPI_WAIT_FOR_COMPLETION(coll.active);
goto done;
}
}
@ -6292,7 +6292,7 @@ static void display_all_timers(int state) {
if( timing_enabled >= 2) {
barrier_start = get_time();
ompi_rte_barrier(&coll);
ORTE_WAIT_FOR_COMPLETION(coll.active);
OMPI_WAIT_FOR_COMPLETION(coll.active);
barrier_stop = get_time();
opal_output(0,
"crcp:bkmrk: timing(%20s): %20s = %10.2f s\n",

130
ompi/mca/dpm/orte/dpm_orte.c
Просмотреть файл

@ -12,7 +12,7 @@
* Copyright (c) 2007-2011 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2006-2009 University of Houston. All rights reserved.
* Copyright (c) 2009 Sun Microsystems, Inc. All rights reserved.
* Copyright (c) 2011-2012 Los Alamos National Security, LLC. All rights
* Copyright (c) 2011-2013 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
@ -30,8 +30,8 @@
#include "opal/util/argv.h"
#include "opal/util/opal_getcwd.h"
#include "opal/dss/dss.h"
#include "orte/mca/errmgr/errmgr.h"
#include "orte/mca/grpcomm/grpcomm.h"
#include "orte/mca/plm/plm.h"
@ -50,6 +50,7 @@
#include "ompi/group/group.h"
#include "ompi/proc/proc.h"
#include "ompi/mca/pml/pml.h"
#include "ompi/mca/rte/rte.h"
#include "ompi/info/info.h"
#include "ompi/mca/dpm/base/base.h"
@ -58,14 +59,6 @@
/* Local static variables */
static opal_mutex_t ompi_dpm_port_mutex;
static orte_rml_tag_t next_tag;
static bool waiting_for_recv = false;
static opal_buffer_t *cabuf=NULL;
static orte_process_name_t carport;
/* Local static functions */
static void recv_cb(int status, orte_process_name_t* sender,
opal_buffer_t *buffer,
orte_rml_tag_t tag, void *cbdata);
/* API functions */
static int init(void);
@ -104,14 +97,6 @@ ompi_dpm_base_module_t ompi_dpm_orte_module = {
finalize
};
static void rml_cbfunc(int status, orte_process_name_t* sender,
opal_buffer_t* buffer, orte_rml_tag_t tag,
void* cbdata)
{
OBJ_RELEASE(buffer);
}
/*
* Init the module
*/
@ -147,6 +132,8 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
orte_grpcomm_collective_t modex;
opal_list_item_t *item;
orte_namelist_t *nm;
orte_rml_recv_cb_t xfer;
orte_process_name_t carport;
OPAL_OUTPUT_VERBOSE((1, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept with port %s %s",
@ -190,6 +177,7 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
opal_progress_event_users_increment();
if ( rank == root ) {
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
if (send_first) {
/* Get a collective id for the modex we need later on - we
* have to get a globally unique id for this purpose as
@ -203,21 +191,23 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
return OMPI_ERROR;
}
/* send the request - doesn't have to include any data */
rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, nbuf, ORTE_RML_TAG_COLL_ID_REQ, 0, rml_cbfunc, NULL);
rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, nbuf,
ORTE_RML_TAG_COLL_ID_REQ,
orte_rml_send_callback, NULL);
/* wait for the id */
waiting_for_recv = true;
cabuf = OBJ_NEW(opal_buffer_t);
rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_COLL_ID,
ORTE_RML_NON_PERSISTENT, recv_cb, NULL);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_COLL_ID,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
/* wait for response */
ORTE_WAIT_FOR_COMPLETION(waiting_for_recv);
xfer.active = true;
OMPI_WAIT_FOR_COMPLETION(xfer.active);
i=1;
if (OPAL_SUCCESS != (rc = opal_dss.unpack(cabuf, &id, &i, ORTE_GRPCOMM_COLL_ID_T))) {
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&xfer.data, &id, &i, ORTE_GRPCOMM_COLL_ID_T))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(cabuf);
OBJ_DESTRUCT(&xfer);
return OMPI_ERROR;
}
OBJ_RELEASE(cabuf);
OBJ_DESTRUCT(&xfer);
/* send it to my peer on the other side */
nbuf = OBJ_NEW(opal_buffer_t);
if (NULL == nbuf) {
@ -227,22 +217,22 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
ORTE_ERROR_LOG(rc);
goto exit;
}
rc = orte_rml.send_buffer_nb(&port, nbuf, tag, 0, rml_cbfunc, NULL);
rc = orte_rml.send_buffer_nb(&port, nbuf, tag, orte_rml_send_callback, NULL);
} else {
/* wait to recv the collective id */
waiting_for_recv = true;
cabuf = OBJ_NEW(opal_buffer_t);
rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT, recv_cb, NULL);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
/* wait for response */
ORTE_WAIT_FOR_COMPLETION(waiting_for_recv);
xfer.active = true;
OMPI_WAIT_FOR_COMPLETION(xfer.active);
i=1;
if (OPAL_SUCCESS != (rc = opal_dss.unpack(cabuf, &id, &i, ORTE_GRPCOMM_COLL_ID_T))) {
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&xfer.data, &id, &i, ORTE_GRPCOMM_COLL_ID_T))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(cabuf);
OBJ_DESTRUCT(&xfer);
return OMPI_ERROR;
}
OBJ_RELEASE(cabuf);
OBJ_DESTRUCT(&xfer);
}
/* Generate the message buffer containing the number of processes and the list of
@ -296,59 +286,56 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
goto exit;
}
if (NULL != cabuf) {
OBJ_RELEASE(cabuf);
}
cabuf = OBJ_NEW(opal_buffer_t);
if (NULL == cabuf ) {
rc = OMPI_ERROR;
goto exit;
}
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
/* Exchange the number and the list of processes in the groups */
if ( send_first ) {
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept sending first to %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&port)));
rc = orte_rml.send_buffer(&port, nbuf, tag, 0);
rc = orte_rml.send_buffer_nb(&port, nbuf, tag, orte_rml_send_callback, NULL);
/* setup to recv */
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept waiting for response",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
waiting_for_recv = true;
rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT, recv_cb, NULL);
xfer.active = true;
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
/* wait for response */
ORTE_WAIT_FOR_COMPLETION(waiting_for_recv);
OMPI_WAIT_FOR_COMPLETION(xfer.active);
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept got data from %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&carport)));
ORTE_NAME_PRINT(&xfer.name)));
} else {
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept recving first",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
/* setup to recv */
waiting_for_recv = true;
rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT, recv_cb, NULL);
xfer.active = true;
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, tag,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
/* wait for response */
ORTE_WAIT_FOR_COMPLETION(waiting_for_recv);
OMPI_WAIT_FOR_COMPLETION(xfer.active);
/* now send our info */
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept sending info to %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&carport)));
rc = orte_rml.send_buffer(&carport, nbuf, tag, 0);
ORTE_NAME_PRINT(&xfer.name)));
rc = orte_rml.send_buffer_nb(&xfer.name, nbuf, tag, orte_rml_send_callback, NULL);
}
if (OPAL_SUCCESS != (rc = opal_dss.unload(cabuf, &rnamebuf, &rnamebuflen))) {
if (OPAL_SUCCESS != (rc = opal_dss.unload(&xfer.data, &rnamebuf, &rnamebuflen))) {
ORTE_ERROR_LOG(rc);
OBJ_DESTRUCT(&xfer.data);
goto exit;
}
carport.jobid = xfer.name.jobid;
carport.vpid = xfer.name.vpid;
OBJ_DESTRUCT(&xfer);
}
/* First convert the size_t to an int so we can cast in the bcast to a void *
@ -504,9 +491,8 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
ORTE_ERROR_LOG(rc);
goto exit;
}
while (modex.active) {
opal_progress();
}
modex.active = true;
OMPI_WAIT_FOR_COMPLETION(modex.active);
OBJ_DESTRUCT(&modex);
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
@ -535,9 +521,6 @@ static int connect_accept ( ompi_communicator_t *comm, int root,
}
OBJ_RELEASE(nrbuf);
if ( rank == root ) {
OBJ_RELEASE(nbuf);
}
OPAL_OUTPUT_VERBOSE((3, ompi_dpm_base_framework.framework_output,
"%s dpm:orte:connect_accept allocating group size %d",
@ -1594,20 +1577,3 @@ static int finalize(void)
OBJ_DESTRUCT(&ompi_dpm_port_mutex);
return OMPI_SUCCESS;
}
static void recv_cb(int status, orte_process_name_t* sender,
opal_buffer_t *buffer,
orte_rml_tag_t tag, void *cbdata)
{
/* copy the payload to the global buffer */
opal_dss.copy_payload(cabuf, buffer);
/* flag the identity of the remote proc */
carport.jobid = sender->jobid;
carport.vpid = sender->vpid;
/* flag complete */
waiting_for_recv = false;
}

16
ompi/mca/pml/bfo/pml_bfo.c
Просмотреть файл

@ -676,7 +676,7 @@ int mca_pml_bfo_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCPBR1);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P0);
@ -688,7 +688,7 @@ int mca_pml_bfo_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_COREBR0);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P2);
}
@ -791,7 +791,7 @@ int mca_pml_bfo_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2PBR1);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P3);
}
@ -810,7 +810,7 @@ int mca_pml_bfo_ft_event( int state )
OBJ_RELEASE(modex);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(modex->active);
OMPI_WAIT_FOR_COMPLETION(modex->active);
OBJ_RELEASE(modex);
/*
@ -827,7 +827,7 @@ int mca_pml_bfo_ft_event( int state )
opal_output(0, "pml:bfo: ft_event(Restart): Failed in ompi_rte_barrier (%d)", ret);
return ret;
}
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
if( NULL != procs ) {
for(p = 0; p < (int)num_procs; ++p) {
@ -841,7 +841,7 @@ int mca_pml_bfo_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2PBR2);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCP1);
}
@ -863,7 +863,7 @@ int mca_pml_bfo_ft_event( int state )
OBJ_RELEASE(modex);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(modex->active);
OMPI_WAIT_FOR_COMPLETION(modex->active);
OBJ_RELEASE(modex);
/*
@ -880,7 +880,7 @@ int mca_pml_bfo_ft_event( int state )
opal_output(0, "pml:bfo: ft_event(Restart): Failed in ompi_rte_barrier (%d)", ret);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
if( NULL != procs ) {
for(p = 0; p < (int)num_procs; ++p) {

16
ompi/mca/pml/ob1/pml_ob1.c
Просмотреть файл

@ -765,7 +765,7 @@ int mca_pml_ob1_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCPBR1);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P0);
@ -777,7 +777,7 @@ int mca_pml_ob1_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_COREBR0);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P2);
}
@ -880,7 +880,7 @@ int mca_pml_ob1_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2PBR1);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2P3);
}
@ -899,7 +899,7 @@ int mca_pml_ob1_ft_event( int state )
OBJ_RELEASE(modex);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(modex->active);
OMPI_WAIT_FOR_COMPLETION(modex->active);
OBJ_RELEASE(modex);
/*
@ -916,7 +916,7 @@ int mca_pml_ob1_ft_event( int state )
opal_output(0, "pml:ob1: ft_event(Restart): Failed in ompi_rte_barrier (%d)", ret);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
if( NULL != procs ) {
for(p = 0; p < (int)num_procs; ++p) {
@ -930,7 +930,7 @@ int mca_pml_ob1_ft_event( int state )
if( opal_cr_timing_barrier_enabled ) {
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_P2PBR2);
ompi_rte_barrier(coll);
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
}
OPAL_CR_SET_TIMER(OPAL_CR_TIMER_CRCP1);
}
@ -952,7 +952,7 @@ int mca_pml_ob1_ft_event( int state )
OBJ_RELEASE(modex);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(modex->active);
OMPI_WAIT_FOR_COMPLETION(modex->active);
OBJ_RELEASE(modex);
/*
@ -969,7 +969,7 @@ int mca_pml_ob1_ft_event( int state )
opal_output(0, "pml:ob1: ft_event(Restart): Failed in ompi_rte_barrier (%d)", ret);
goto clean;
}
ORTE_WAIT_FOR_COMPLETION(coll->active);
OMPI_WAIT_FOR_COMPLETION(coll->active);
if( NULL != procs ) {
for(p = 0; p < (int)num_procs; ++p) {

120
ompi/mca/pubsub/orte/pubsub_orte.c
Просмотреть файл

@ -10,8 +10,9 @@
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2012 Los Alamos National Security, LLC. All rights
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -40,6 +41,7 @@
#include "orte/runtime/orte_data_server.h"
#include "ompi/info/info.h"
#include "ompi/mca/rte/rte.h"
#include "ompi/mca/pubsub/base/base.h"
#include "pubsub_orte.h"
@ -129,9 +131,10 @@ static int publish ( char *service_name, ompi_info_t *info, char *port_name )
int rc, ret, flag;
bool global_scope = false;
orte_process_name_t *info_host;
opal_buffer_t buf;
opal_buffer_t *buf;
orte_data_server_cmd_t cmd=ORTE_DATA_SERVER_PUBLISH;
orte_std_cntr_t cnt;
orte_rml_recv_cb_t xfer;
bool unique=false;
ompi_info_get_bool(info, "ompi_global_scope", &global_scope, &flag);
@ -180,54 +183,56 @@ static int publish ( char *service_name, ompi_info_t *info, char *port_name )
}
/* construct the buffer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
buf = OBJ_NEW(opal_buffer_t);
/* pack the publish command */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
/* pack the service name */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &service_name, 1, OPAL_STRING))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &service_name, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
/* pack the port name */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &port_name, 1, OPAL_STRING))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &port_name, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
/* pack the uniqueness flag */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &unique, 1, OPAL_BOOL))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &unique, 1, OPAL_BOOL))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
/* send the data */
if (0 > (rc = orte_rml.send_buffer(info_host, &buf, ORTE_RML_TAG_DATA_SERVER, 0))) {
if (0 > (rc = orte_rml.send_buffer_nb(info_host, buf,
ORTE_RML_TAG_DATA_SERVER,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
OBJ_DESTRUCT(&buf);
/* get the answer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
if (0 > (rc = orte_rml.recv_buffer(ORTE_NAME_WILDCARD, &buf, ORTE_RML_TAG_DATA_CLIENT, 0))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DATA_CLIENT,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
xfer.active = true;
OMPI_WAIT_FOR_COMPLETION(xfer.active);
/* unpack the result */
cnt = 1;
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&buf, &ret, &cnt, OPAL_INT))) {
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&xfer.data, &ret, &cnt, OPAL_INT))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
rc = ret;
OBJ_DESTRUCT(&xfer);
CLEANUP:
OBJ_DESTRUCT(&buf);
@ -240,7 +245,7 @@ enum { NONE, LOCAL, GLOBAL };
static char* lookup ( char *service_name, ompi_info_t *info )
{
orte_process_name_t *info_host;
opal_buffer_t buf;
opal_buffer_t *buf;
orte_data_server_cmd_t cmd=ORTE_DATA_SERVER_LOOKUP;
orte_std_cntr_t cnt=0;
char *port_name=NULL;
@ -248,6 +253,7 @@ static char* lookup ( char *service_name, ompi_info_t *info )
char value[256], **tokens, *ptr;
int lookup[2] = { GLOBAL, LOCAL };
size_t num_tokens;
orte_rml_recv_cb_t xfer;
/* Look in the MPI_Info (ompi_info_t*) for the key
* "ompi_lookup_order". Acceptable values are:
@ -368,37 +374,40 @@ static char* lookup ( char *service_name, ompi_info_t *info )
/* go look it up */
/* construct the buffer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
buf = OBJ_NEW(opal_buffer_t);
/* pack the lookup command */
if (OPAL_SUCCESS != (ret = opal_dss.pack(&buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
if (OPAL_SUCCESS != (ret = opal_dss.pack(buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
ORTE_ERROR_LOG(ret);
goto CLEANUP;
}
/* pack the service name */
if (OPAL_SUCCESS != (ret = opal_dss.pack(&buf, &service_name, 1, OPAL_STRING))) {
if (OPAL_SUCCESS != (ret = opal_dss.pack(buf, &service_name, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(ret);
goto CLEANUP;
}
/* send the cmd */
if (0 > (ret = orte_rml.send_buffer(info_host, &buf, ORTE_RML_TAG_DATA_SERVER, 0))) {
if (0 > (ret = orte_rml.send_buffer_nb(info_host, buf,
ORTE_RML_TAG_DATA_SERVER,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(ret);
goto CLEANUP;
}
OBJ_DESTRUCT(&buf);
/* get the answer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
if (0 > (ret = orte_rml.recv_buffer(ORTE_NAME_WILDCARD, &buf, ORTE_RML_TAG_DATA_CLIENT, 0))) {
ORTE_ERROR_LOG(ret);
goto CLEANUP;
}
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DATA_CLIENT,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
xfer.active = true;
OMPI_WAIT_FOR_COMPLETION(xfer.active);
/* unpack the return code */
cnt = 1;
if (OPAL_SUCCESS != (ret = opal_dss.unpack(&buf, &rc, &cnt, OPAL_INT))) {
if (OPAL_SUCCESS != (ret = opal_dss.unpack(&xfer.data, &rc, &cnt, OPAL_INT))) {
ORTE_ERROR_LOG(ret);
goto CLEANUP;
}
@ -410,8 +419,9 @@ static char* lookup ( char *service_name, ompi_info_t *info )
if (ORTE_SUCCESS == rc) {
/* the server was able to lookup the port - unpack the port name */
cnt=1;
if (OPAL_SUCCESS != (ret = opal_dss.unpack(&buf, &port_name, &cnt, OPAL_STRING))) {
if (OPAL_SUCCESS != (ret = opal_dss.unpack(&xfer.data, &port_name, &cnt, OPAL_STRING))) {
ORTE_ERROR_LOG(ret);
OBJ_DESTRUCT(&xfer);
goto CLEANUP;
}
@ -422,25 +432,20 @@ static char* lookup ( char *service_name, ompi_info_t *info )
if (NULL != port_name) {
/* got an answer - return it */
OBJ_DESTRUCT(&buf);
OBJ_DESTRUCT(&xfer);
return port_name;
}
}
/* if we didn't get a port_name, then continue */
OBJ_DESTRUCT(&buf);
OBJ_DESTRUCT(&xfer);
}
/* only get here if we tried both options and failed - since the
* buffer will already have been cleaned up, just return
*/
return NULL;
CLEANUP:
OBJ_DESTRUCT(&buf);
return NULL;
}
/*
@ -453,10 +458,11 @@ static int unpublish ( char *service_name, ompi_info_t *info )
int rc, ret, flag;
bool global_scope;
orte_process_name_t *info_host;
opal_buffer_t buf;
opal_buffer_t *buf;
orte_data_server_cmd_t cmd=ORTE_DATA_SERVER_UNPUBLISH;
orte_std_cntr_t cnt;
orte_rml_recv_cb_t xfer;
ompi_info_get_bool(info, "ompi_global_scope", &global_scope, &flag);
if (0 == flag) {
@ -497,45 +503,49 @@ static int unpublish ( char *service_name, ompi_info_t *info )
service_name, global_scope ? "Global" : "Local"));
/* construct the buffer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
buf = OBJ_NEW(opal_buffer_t);
/* pack the unpublish command */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &cmd, 1, ORTE_DATA_SERVER_CMD))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
goto CLEANUP;
}
/* pack the service name */
if (OPAL_SUCCESS != (rc = opal_dss.pack(&buf, &service_name, 1, OPAL_STRING))) {
if (OPAL_SUCCESS != (rc = opal_dss.pack(buf, &service_name, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
goto CLEANUP;
}
/* send the command */
if (0 > (rc = orte_rml.send_buffer(info_host, &buf, ORTE_RML_TAG_DATA_SERVER, 0))) {
if (0 > (rc = orte_rml.send_buffer_nb(info_host, buf, ORTE_RML_TAG_DATA_SERVER,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
goto CLEANUP;
}
OBJ_DESTRUCT(&buf);
/* get the answer */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
if (0 > (rc = orte_rml.recv_buffer(ORTE_NAME_WILDCARD, &buf, ORTE_RML_TAG_DATA_CLIENT, 0))) {
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_DATA_CLIENT,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
xfer.active = true;
OMPI_WAIT_FOR_COMPLETION(xfer.active);
/* unpack the result */
cnt = 1;
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&buf, &ret, &cnt, OPAL_INT))) {
if (OPAL_SUCCESS != (rc = opal_dss.unpack(&xfer.data, &ret, &cnt, OPAL_INT))) {
ORTE_ERROR_LOG(rc);
OBJ_DESTRUCT(&xfer);
goto CLEANUP;
}
OBJ_DESTRUCT(&xfer);
rc = ret;
CLEANUP:
OBJ_DESTRUCT(&buf);
CLEANUP:
return rc;
}

5
ompi/mca/rte/base/base.h
Просмотреть файл

@ -24,11 +24,6 @@
BEGIN_C_DECLS
/*
* MCA Framework
*/
OMPI_DECLSPEC extern mca_base_framework_t ompi_rte_base_framework;
/**
* Select an available component.
*

8
ompi/mca/rte/orte/rte_orte.h
Просмотреть файл

@ -102,12 +102,11 @@ OMPI_DECLSPEC int ompi_rte_db_remove(const ompi_process_name_t *nm,
/* Communications */
typedef orte_rml_tag_t ompi_rml_tag_t;
#define ompi_rte_send_buffer(a, b, c, d) orte_rml.send_buffer(a, b, c, d)
#define ompi_rte_send_buffer_nb(a, b, c, d, e, f) orte_rml.send_buffer_nb(a, b, c, d, e, f)
#define ompi_rte_recv_buffer(a, b, c, d) orte_rml.recv_buffer(a, b, c, d)
#define ompi_rte_send_buffer_nb(a, b, c, d, e) orte_rml.send_buffer_nb(a, b, c, d, e)
#define ompi_rte_recv_buffer_nb(a, b, c, d, e) orte_rml.recv_buffer_nb(a, b, c, d, e)
#define ompi_rte_recv_cancel(a, b) orte_rml.recv_cancel(a, b)
#define ompi_rte_parse_uris(a, b, c) orte_rml_base_parse_uris(a, b, c)
#define ompi_rte_send_cbfunc orte_rml_send_callback
/* Communication tags */
/* carry over the INVALID def */
@ -115,7 +114,8 @@ typedef orte_rml_tag_t ompi_rml_tag_t;
/* define a starting point to avoid conflicts */
#define OMPI_RML_TAG_BASE ORTE_RML_TAG_MAX
#define OMPI_RML_PERSISTENT ORTE_RML_PERSISTENT
#define OMPI_RML_PERSISTENT ORTE_RML_PERSISTENT
#define OMPI_RML_NON_PERSISTENT ORTE_RML_NON_PERSISTENT
END_C_DECLS

25
ompi/mca/rte/orte/rte_orte_module.c
Просмотреть файл

@ -78,7 +78,6 @@ void ompi_rte_abort(int error_code, char *fmt, ...)
/* No way to reach here */
}
/*
* Wait for a debugger if asked. We support two ways of waiting for
* attaching debuggers -- see big comment in
@ -86,8 +85,8 @@ void ompi_rte_abort(int error_code, char *fmt, ...)
*/
void ompi_rte_wait_for_debugger(void)
{
opal_buffer_t buf;
int debugger, rc;
int debugger;
orte_rml_recv_cb_t xfer;
/* See lengthy comment in orte/tools/orterun/debuggers.c about
orte_in_parallel_debugger */
@ -122,22 +121,18 @@ void ompi_rte_wait_for_debugger(void)
* spin in * the grpcomm barrier in ompi_mpi_init until rank=0
* joins them.
*/
if (0 != OMPI_PROC_MY_NAME->vpid) {
if (0 != ORTE_PROC_MY_NAME->vpid) {
return;
}
/* VPID 0 waits for a message from the HNP */
OBJ_CONSTRUCT(&buf, opal_buffer_t);
rc = ompi_rte_recv_buffer(OMPI_NAME_WILDCARD, &buf,
ORTE_RML_TAG_DEBUGGER_RELEASE, 0);
OBJ_DESTRUCT(&buf); /* don't care about contents of message */
if (rc < 0) {
/* if it failed for some reason, then we are in trouble -
* for now, just report the problem and give up waiting
*/
opal_output(0, "Debugger_attach[rank=%ld]: could not wait for debugger!",
(long)OMPI_PROC_MY_NAME->vpid);
}
OBJ_CONSTRUCT(&xfer, orte_rml_recv_cb_t);
orte_rml.recv_buffer_nb(OMPI_NAME_WILDCARD,
ORTE_RML_TAG_DEBUGGER_RELEASE,
ORTE_RML_NON_PERSISTENT,
orte_rml_recv_callback, &xfer);
xfer.active = true;
ORTE_WAIT_FOR_COMPLETION(xfer.active);
}
}

22
ompi/mca/rte/rte.h
Просмотреть файл

@ -210,6 +210,28 @@ BEGIN_C_DECLS
#define OMPI_RML_TAG_DYNAMIC OMPI_RML_TAG_BASE+200
/*
* MCA Framework
*/
OMPI_DECLSPEC extern mca_base_framework_t ompi_rte_base_framework;
/* In a few places, we need to barrier until something happens
* that changes a flag to indicate we can release - e.g., waiting
* for a specific RTE message to arrive. We don't want to block MPI
* progress while waiting, so we loop over opal_progress, letting
* the RTE progress thread move the RTE along
*/
#define OMPI_WAIT_FOR_COMPLETION(flg) \
do { \
opal_output_verbose(1, ompi_rte_base_framework.framework_output, \
"%s waiting on RTE event at %s:%d", \
OMPI_NAME_PRINT(OMPI_PROC_MY_NAME), \
__FILE__, __LINE__); \
while ((flg)) { \
opal_progress(); \
} \
}while(0);
typedef struct {
opal_list_item_t super;
ompi_process_name_t name;

12
ompi/mca/vprotocol/pessimist/vprotocol_pessimist_eventlog.c
Просмотреть файл

@ -19,7 +19,7 @@
int vprotocol_pessimist_event_logger_connect(int el_rank, ompi_communicator_t **el_comm)
{
int rc;
opal_buffer_t buffer;
opal_buffer_t *buffer;
char *port;
ompi_process_name_t el_proc;
char *hnp_uri, *rml_uri;
@ -57,14 +57,8 @@ int vprotocol_pessimist_event_logger_connect(int el_rank, ompi_communicator_t **
/* Send an rml message to tell the remote end to wake up and jump into
* connect/accept */
OBJ_CONSTRUCT(&buffer, opal_buffer_t);
rc = ompi_rte_send_buffer(&el_proc, &buffer, el_tag+1, 0);
if(OMPI_SUCCESS > rc) {
OMPI_ERROR_LOG(rc);
OBJ_DESTRUCT(&buffer);
return rc;
}
OBJ_DESTRUCT(&buffer);
buffer = OBJ_NEW(opal_buffer_t);
ompi_rte_send_buffer_nb(&el_proc, buffer, el_tag+1, NULL, NULL);
rc = ompi_dpm.connect_accept(MPI_COMM_SELF, 0, port, true, el_comm);
if(OMPI_SUCCESS != rc) {

5
ompi/runtime/ompi_mpi_finalize.c
Просмотреть файл

@ -207,15 +207,14 @@ int ompi_mpi_finalize(void)
*/
coll = OBJ_NEW(ompi_rte_collective_t);
coll->id = ompi_process_info.peer_fini_barrier;
coll->active = true;
if (OMPI_SUCCESS != (ret = ompi_rte_barrier(coll))) {
OMPI_ERROR_LOG(ret);
return ret;
}
/* wait for barrier to complete */
while (coll->active) {
opal_progress(); /* block in progress pending events */
}
OMPI_WAIT_FOR_COMPLETION(coll->active);
OBJ_RELEASE(coll);
/* check for timing request - get stop time and report elapsed

10
ompi/runtime/ompi_mpi_init.c
Просмотреть файл

@ -612,6 +612,7 @@ int ompi_mpi_init(int argc, char **argv, int requested, int *provided)
*/
coll = OBJ_NEW(ompi_rte_collective_t);
coll->id = ompi_process_info.peer_modex;
coll->active = true;
if (OMPI_SUCCESS != (ret = ompi_rte_modex(coll))) {
error = "rte_modex failed";
goto error;
@ -620,9 +621,7 @@ int ompi_mpi_init(int argc, char **argv, int requested, int *provided)
* so long as it occurs prior to calling a function that needs
* the modex info!
*/
while (coll->active) {
opal_progress(); /* block in progress pending events */
}
OMPI_WAIT_FOR_COMPLETION(coll->active);
OBJ_RELEASE(coll);
if (ompi_enable_timing && 0 == OMPI_PROC_MY_NAME->vpid) {
@ -802,14 +801,13 @@ int ompi_mpi_init(int argc, char **argv, int requested, int *provided)
/* wait for everyone to reach this point */
coll = OBJ_NEW(ompi_rte_collective_t);
coll->id = ompi_process_info.peer_init_barrier;
coll->active = true;
if (OMPI_SUCCESS != (ret = ompi_rte_barrier(coll))) {
error = "rte_barrier failed";
goto error;
}
/* wait for barrier to complete */
while (coll->active) {
opal_progress(); /* block in progress pending events */
}
OMPI_WAIT_FOR_COMPLETION(coll->active);
OBJ_RELEASE(coll);
/* check for timing request - get stop time and report elapsed

5
ompi/tools/ompi_info/param.c
Просмотреть файл

@ -262,13 +262,12 @@ void ompi_info_do_config(bool want_all)
if (OPAL_HAVE_SOLARIS_THREADS || OPAL_HAVE_POSIX_THREADS) { /* should just test OPAL_HAVE_THREADS */
#if OMPI_RTE_ORTE
(void)asprintf(&threads, "%s (MPI_THREAD_MULTIPLE: %s, OPAL support: %s, OMPI progress: %s, ORTE progress: %s, Event lib: yes)",
(void)asprintf(&threads, "%s (MPI_THREAD_MULTIPLE: %s, OPAL support: %s, OMPI progress: %s, ORTE progress: yes, Event lib: yes)",
OPAL_HAVE_SOLARIS_THREADS ? "solaris" :
(OPAL_HAVE_POSIX_THREADS ? "posix" : "type unknown"), /* "type unknown" can presumably never happen */
OMPI_ENABLE_THREAD_MULTIPLE ? "yes" : "no",
OPAL_ENABLE_MULTI_THREADS ? "yes" : "no",
OMPI_ENABLE_PROGRESS_THREADS ? "yes" : "no",
ORTE_ENABLE_PROGRESS_THREADS ? "yes" : "no");
OMPI_ENABLE_PROGRESS_THREADS ? "yes" : "no");
#else
(void)asprintf(&threads, "%s (MPI_THREAD_MULTIPLE: %s, OPAL support: %s, OMPI progress: %s, Event lib: yes)",
OPAL_HAVE_SOLARIS_THREADS ? "solaris" :

30
opal/dss/dss_load_unload.c
Просмотреть файл

@ -27,9 +27,6 @@
int opal_dss_unload(opal_buffer_t *buffer, void **payload,
int32_t *bytes_used)
{
char *hdr_dst = NULL;
opal_dss_buffer_type_t type;
/* check that buffer is not null */
if (!buffer) {
return OPAL_ERR_BAD_PARAM;
@ -47,18 +44,6 @@ int opal_dss_unload(opal_buffer_t *buffer, void **payload,
return OPAL_SUCCESS;
}
/* add room for our description of the buffer -- currently just the type */
if (NULL == (hdr_dst = opal_dss_buffer_extend(buffer,
sizeof(opal_dss_buffer_type_t)))) {
return OPAL_ERR_OUT_OF_RESOURCE;
}
/* add the header (at the end, so perhaps it's a footer? */
type = buffer->type;
OPAL_DSS_BUFFER_TYPE_HTON(type);
memcpy(hdr_dst, &type, sizeof(opal_dss_buffer_type_t));
buffer->bytes_used += sizeof(opal_dss_buffer_type_t);
/* okay, we have something to provide - pass it back */
*payload = buffer->base_ptr;
*bytes_used = buffer->bytes_used;
@ -77,31 +62,16 @@ int opal_dss_unload(opal_buffer_t *buffer, void **payload,
int opal_dss_load(opal_buffer_t *buffer, void *payload,
int32_t bytes_used)
{
char *hdr_ptr;
opal_dss_buffer_type_t type;
/* check to see if the buffer has been initialized */
if (NULL == buffer) {
return OPAL_ERR_BAD_PARAM;
}
/* check that the payload is there */
if (NULL == payload) {
return OPAL_SUCCESS;
}
/* check if buffer already has payload - free it if so */
if (NULL != buffer->base_ptr) {
free(buffer->base_ptr);
}
/* get our header */
hdr_ptr = (char*) payload + bytes_used - sizeof(opal_dss_buffer_type_t);
memcpy(&type, hdr_ptr, sizeof(opal_dss_buffer_type_t));
OPAL_DSS_BUFFER_TYPE_NTOH(type);
buffer->type = type;
bytes_used -= sizeof(opal_dss_buffer_type_t);
/* populate the buffer */
buffer->base_ptr = (char*)payload;

2
opal/include/opal/types.h
Просмотреть файл

@ -65,7 +65,9 @@ typedef union {
#if defined(__APPLE__) || defined(__WINDOWS__)
typedef char* ompi_iov_base_ptr_t;
#define OPAL_IOVBASE char
#else
#define OPAL_IOVBASE void
typedef void* ompi_iov_base_ptr_t;
#endif

4
opal/include/opal_config_bottom.h
Просмотреть файл

@ -467,9 +467,9 @@ static inline uint16_t ntohs(uint16_t netvar) { return netvar; }
* this makes it simpler to check throughout the code base.
*/
#if OPAL_ENABLE_IPV6 && defined(HAVE_STRUCT_SOCKADDR_IN6)
#define OPAL_WANT_IPV6 1
#define OPAL_ENABLE_IPV6 1
#else
#define OPAL_WANT_IPV6 0
#define OPAL_ENABLE_IPV6 0
#endif
#if !defined(HAVE_STRUCT_SOCKADDR_STORAGE) && defined(HAVE_STRUCT_SOCKADDR_IN)

6
opal/mca/db/base/db_base_fns.c
Просмотреть файл

@ -128,14 +128,12 @@ int opal_db_base_fetch(const opal_identifier_t *proc,
* the operation - anything else is a true error.
*/
if (OPAL_ERR_TAKE_NEXT_OPTION != rc) {
OPAL_ERROR_LOG(rc);
return rc;
}
}
/* if we get here without performing the operation, that's an error */
if (!did_op) {
OPAL_ERROR_LOG(OPAL_ERR_DATA_VALUE_NOT_FOUND);
return OPAL_ERR_DATA_VALUE_NOT_FOUND;
}
return OPAL_SUCCESS;
@ -163,14 +161,12 @@ int opal_db_base_fetch_pointer(const opal_identifier_t *proc,
* the operation - anything else is a true error.
*/
if (OPAL_ERR_TAKE_NEXT_OPTION != rc) {
OPAL_ERROR_LOG(rc);
return rc;
}
}
/* if we get here without performing the operation, that's an error */
if (!did_op) {
OPAL_ERROR_LOG(OPAL_ERR_DATA_VALUE_NOT_FOUND);
return OPAL_ERR_DATA_VALUE_NOT_FOUND;
}
return OPAL_SUCCESS;
@ -198,14 +194,12 @@ int opal_db_base_fetch_multiple(const opal_identifier_t *proc,
* the operation - anything else is a true error.
*/
if (OPAL_ERR_TAKE_NEXT_OPTION != rc) {
OPAL_ERROR_LOG(rc);
return rc;
}
}
/* if we get here without performing the operation, that's an error */
if (!did_op) {
OPAL_ERROR_LOG(OPAL_ERR_DATA_VALUE_NOT_FOUND);
return OPAL_ERR_DATA_VALUE_NOT_FOUND;
}
return OPAL_SUCCESS;

24
opal/mca/db/pmi/db_pmi.c
Просмотреть файл

@ -223,9 +223,6 @@ static int pmi_store_encoded(const opal_identifier_t *uid, const char *key, cons
data_len = bo->size;
}
/* need to bump up the needed if the type ever gets larger than 255 */
assert (type < (1 << 8) && data_len < 0xffff);
needed = 10 + data_len + strlen (key);
if (NULL == pmi_packed_data) {
@ -260,6 +257,10 @@ static int pmi_get_packed (const opal_identifier_t *uid, char **packed_data, siz
opal_identifier_t proc;
int rc;
/* set default */
*packed_data = NULL;
*len = 0;
/* to protect alignment, copy the data across */
memcpy(&proc, uid, sizeof(opal_identifier_t));
@ -306,15 +307,18 @@ static int pmi_get_packed (const opal_identifier_t *uid, char **packed_data, siz
}
}
OPAL_OUTPUT_VERBOSE((10, opal_db_base_framework.framework_output,
"Read data %s\n", tmp_encoded));
free (pmi_tmp);
*packed_data = (char *) pmi_decode (tmp_encoded, len);
free (tmp_encoded);
if (NULL == *packed_data) {
return OPAL_ERR_OUT_OF_RESOURCE;
OPAL_OUTPUT_VERBOSE((10, opal_db_base_framework.framework_output,
"Read data %s\n",
(NULL == tmp_encoded) ? "NULL" : tmp_encoded));
if (NULL != tmp_encoded) {
*packed_data = (char *) pmi_decode (tmp_encoded, len);
free (tmp_encoded);
if (NULL == *packed_data) {
return OPAL_ERR_OUT_OF_RESOURCE;
}
}
return OPAL_SUCCESS;

1
opal/mca/if/base/if_base_components.c
Просмотреть файл

@ -79,6 +79,7 @@ static void opal_if_construct(opal_if_t *obj)
memset(obj->if_name, 0, sizeof(obj->if_name));
obj->if_index = -1;
obj->if_kernel_index = (uint16_t) -1;
obj->af_family = PF_UNSPEC;
obj->if_flags = 0;
obj->if_speed = 0;
memset(&obj->if_addr, 0, sizeof(obj->if_addr));

1
opal/mca/if/bsdx_ipv4/if_bsdx.c
Просмотреть файл

@ -120,6 +120,7 @@ static int if_bsdx_open(void)
(int) sizeof(opal_if_t));
return OPAL_ERR_OUT_OF_RESOURCE;
}
intf->af_family = AF_INET;
/* fill values into the opal_if_t */
memcpy(&a4, &(sin_addr->sin_addr), sizeof(struct in_addr));

23
opal/mca/if/bsdx_ipv6/if_bsdx_ipv6.c
Просмотреть файл

@ -100,7 +100,7 @@ opal_if_base_component_t mca_if_bsdx_ipv6_component = {
/* configure using getifaddrs(3) */
static int if_bsdx_ipv6_open(void)
{
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
struct ifaddrs **ifadd_list;
struct ifaddrs *cur_ifaddrs;
struct sockaddr_in6* sin_addr;
@ -145,6 +145,8 @@ static int if_bsdx_ipv6_open(void)
/* skip interface if it is a loopback device (IFF_LOOPBACK set) */
if (!opal_if_retain_loopback && 0 != (cur_ifaddrs->ifa_flags & IFF_LOOPBACK)) {
opal_output_verbose(1, opal_if_base_framework.framework_output,
"skipping loopback interface %s.\n", cur_ifaddrs->ifa_name);
continue;
}
@ -152,18 +154,11 @@ static int if_bsdx_ipv6_open(void)
/* TODO: do we really skip p2p? */
if (0!= (cur_ifaddrs->ifa_flags & IFF_POINTOPOINT)) {
opal_output_verbose(1, opal_if_base_framework.framework_output,
"skipping loopback interface %s.\n", cur_ifaddrs->ifa_name);
"skipping p2p interface %s.\n", cur_ifaddrs->ifa_name);
continue;
}
sin_addr = (struct sockaddr_in6 *) cur_ifaddrs->ifa_addr;
intf = OBJ_NEW(opal_if_t);
if (NULL == intf) {
opal_output(0, "opal_ifinit: unable to allocate %lu bytes\n",
sizeof(opal_if_t));
free(ifadd_list);
return OPAL_ERR_OUT_OF_RESOURCE;
}
/*
* skip IPv6 address starting with fe80:, as this is supposed to be
@ -195,6 +190,14 @@ static int if_bsdx_ipv6_open(void)
/* fill values into the opal_if_t */
memcpy(&a6, &(sin_addr->sin6_addr), sizeof(struct in6_addr));
intf = OBJ_NEW(opal_if_t);
if (NULL == intf) {
opal_output(0, "opal_ifinit: unable to allocate %lu bytes\n",
sizeof(opal_if_t));
free(ifadd_list);
return OPAL_ERR_OUT_OF_RESOURCE;
}
intf->af_family = AF_INET6;
strncpy(intf->if_name, cur_ifaddrs->ifa_name, IF_NAMESIZE);
intf->if_index = opal_list_get_size(&opal_if_list) + 1;
((struct sockaddr_in6*) &intf->if_addr)->sin6_addr = a6;
@ -220,7 +223,7 @@ static int if_bsdx_ipv6_open(void)
} /* of for loop over ifaddrs list */
free(ifadd_list);
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
return OPAL_SUCCESS;
}

1
opal/mca/if/if.h
Просмотреть файл

@ -88,6 +88,7 @@ typedef struct opal_if_t {
char if_name[IF_NAMESIZE];
int if_index;
uint16_t if_kernel_index;
uint16_t af_family;
int if_flags;
int if_speed;
struct sockaddr_storage if_addr;

31
opal/mca/if/linux_ipv6/if_linux_ipv6.c
Просмотреть файл

@ -63,6 +63,7 @@
#include "opal/util/if.h"
#include "opal/util/output.h"
#include "opal/mca/if/if.h"
#include "opal/mca/if/base/base.h"
static int if_linux_ipv6_open(void);
@ -92,7 +93,7 @@ opal_if_base_component_t mca_if_linux_ipv6_component = {
/* configure using getifaddrs(3) */
static int if_linux_ipv6_open(void)
{
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
FILE *f;
if ((f = fopen("/proc/net/if_inet6", "r"))) {
char ifname[IF_NAMESIZE];
@ -110,8 +111,21 @@ static int if_linux_ipv6_open(void)
&idx, &pfxlen, &scope, &dadstat, ifname) != EOF) {
opal_if_t *intf;
/* we don't want any other scope than global */
if (scope != 0) {
opal_output_verbose(1, opal_if_base_framework.framework_output,
"found interface %2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x scope %x\n",
addrbyte[0], addrbyte[1], addrbyte[2], addrbyte[3],
addrbyte[4], addrbyte[5], addrbyte[6], addrbyte[7],
addrbyte[8], addrbyte[9], addrbyte[10], addrbyte[11],
addrbyte[12], addrbyte[13], addrbyte[14], addrbyte[15], scope);
/* we don't want any other scope less than link-local */
if (scope < 0x20) {
opal_output_verbose(1, opal_if_base_framework.framework_output,
"skipping interface %2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x scope %x\n",
addrbyte[0], addrbyte[1], addrbyte[2], addrbyte[3],
addrbyte[4], addrbyte[5], addrbyte[6], addrbyte[7],
addrbyte[8], addrbyte[9], addrbyte[10], addrbyte[11],
addrbyte[12], addrbyte[13], addrbyte[14], addrbyte[15], scope);
continue;
}
@ -122,7 +136,8 @@ static int if_linux_ipv6_open(void)
fclose(f);
return OPAL_ERR_OUT_OF_RESOURCE;
}
intf->af_family = AF_INET6;
for (iter = 0; iter < 16; iter++) {
a6.s6_addr[iter] = addrbyte[iter];
}
@ -144,10 +159,16 @@ static int if_linux_ipv6_open(void)
/* copy new interface information to heap and append
to list */
opal_list_append(&opal_if_list, &(intf->super));
opal_output_verbose(1, opal_if_base_framework.framework_output,
"added interface %2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x:%2x%2x\n",
addrbyte[0], addrbyte[1], addrbyte[2], addrbyte[3],
addrbyte[4], addrbyte[5], addrbyte[6], addrbyte[7],
addrbyte[8], addrbyte[9], addrbyte[10], addrbyte[11],
addrbyte[12], addrbyte[13], addrbyte[14], addrbyte[15]);
} /* of while */
fclose(f);
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
return OPAL_SUCCESS;
}

1
opal/mca/if/posix_ipv4/if_posix.c
Просмотреть файл

@ -211,6 +211,7 @@ static int if_posix_open(void)
close(sd);
return OPAL_ERR_OUT_OF_RESOURCE;
}
intf->af_family = AF_INET;
/* copy entry over into our data structure */
memset(intf->if_name, 0, sizeof(intf->if_name));

5
opal/mca/if/solaris_ipv6/if_solaris_ipv6.c
Просмотреть файл

@ -45,7 +45,7 @@ opal_if_base_component_t mca_if_solaris_ipv6_component = {
/* configure using getifaddrs(3) */
static int if_solaris_ipv6_open(void)
{
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
int i;
int sd;
int error;
@ -138,6 +138,7 @@ static int if_solaris_ipv6_open(void)
sizeof (opal_if_t));
return OPAL_ERR_OUT_OF_RESOURCE;
}
intf->af_family = AF_INET6;
strncpy (intf->if_name, lifreq->lifr_name, IF_NAMESIZE);
intf->if_index = opal_list_get_size(&opal_if_list)+1;
@ -156,7 +157,7 @@ static int if_solaris_ipv6_open(void)
if (NULL != lifconf.lifc_buf) {
free (lifconf.lifc_buf);
}
#endif /* OPAL_WANT_IPV6 */
#endif /* OPAL_ENABLE_IPV6 */
return OPAL_SUCCESS;
}

50
opal/util/if.c
Просмотреть файл

@ -191,14 +191,8 @@ int opal_ifindextokindex(int if_index)
int opal_ifaddrtoname(const char* if_addr, char* if_name, int length)
{
opal_if_t* intf;
#if OPAL_WANT_IPV6
int error;
struct addrinfo hints, *res = NULL, *r;
#else
int i;
in_addr_t inaddr;
struct hostent *h;
#endif
/* if the user asked us not to resolve interfaces, then just return */
if (opal_if_do_not_resolve) {
@ -212,7 +206,6 @@ int opal_ifaddrtoname(const char* if_addr, char* if_name, int length)
return OPAL_ERROR;
}
#if OPAL_WANT_IPV6
memset(&hints, 0, sizeof(hints));
hints.ai_family = PF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
@ -241,36 +234,23 @@ int opal_ifaddrtoname(const char* if_addr, char* if_name, int length)
strncpy(if_name, intf->if_name, length);
return OPAL_SUCCESS;
}
} else {
}
#if OPAL_ENABLE_IPV6
else {
if (IN6_ARE_ADDR_EQUAL(&((struct sockaddr_in6*) &intf->if_addr)->sin6_addr,
&((struct sockaddr_in6*) r->ai_addr)->sin6_addr)) {
strncpy(if_name, intf->if_name, length);
return OPAL_SUCCESS;
}
}
#endif
}
}
if (NULL != res) {
freeaddrinfo (res);
}
#else
h = gethostbyname(if_addr);
if (0 == h) {
return OPAL_ERR_NOT_FOUND;
}
for (i=0; NULL != h->h_addr_list[i]; i++) {
memcpy(&inaddr, h->h_addr_list[i], sizeof(inaddr));
for (intf = (opal_if_t*)opal_list_get_first(&opal_if_list);
intf != (opal_if_t*)opal_list_get_end(&opal_if_list);
intf = (opal_if_t*)opal_list_get_next(intf)) {
if (((struct sockaddr_in*) &intf->if_addr)->sin_addr.s_addr == inaddr) {
strncpy(if_name, intf->if_name, length);
return OPAL_SUCCESS;
}
}
}
#endif
/* if we get here, it wasn't found */
return OPAL_ERR_NOT_FOUND;
}
@ -284,6 +264,7 @@ int16_t opal_ifaddrtokindex(const char* if_addr)
opal_if_t* intf;
int error;
struct addrinfo hints, *res = NULL, *r;
size_t len;
if (OPAL_SUCCESS != mca_base_framework_open(&opal_if_base_framework, 0)) {
return OPAL_ERROR;
@ -306,18 +287,21 @@ int16_t opal_ifaddrtokindex(const char* if_addr)
intf != (opal_if_t*)opal_list_get_end(&opal_if_list);
intf = (opal_if_t*)opal_list_get_next(intf)) {
if (AF_INET == r->ai_family) {
if (AF_INET == r->ai_family && AF_INET == intf->af_family) {
struct sockaddr_in ipv4;
memcpy (&ipv4, r->ai_addr, r->ai_addrlen);
len = (r->ai_addrlen < sizeof(struct sockaddr_in)) ? r->ai_addrlen : sizeof(struct sockaddr_in);
memcpy(&ipv4, r->ai_addr, len);
if (opal_net_samenetwork((struct sockaddr*)&ipv4, (struct sockaddr*)&intf->if_addr, intf->if_mask)) {
return intf->if_kernel_index;
}
}
#if OPAL_WANT_IPV6
else {
if (opal_net_samenetwork((struct sockaddr*)&((struct sockaddr_in6*)&intf->if_addr)->sin6_addr,
(struct sockaddr*)&((struct sockaddr_in6*) r->ai_addr)->sin6_addr, intf->if_mask)) {
#if OPAL_ENABLE_IPV6
else if (AF_INET6 == r->ai_family && AF_INET6 == intf->af_family) {
struct sockaddr_in6 ipv6;
len = (r->ai_addrlen < sizeof(struct sockaddr_in6)) ? r->ai_addrlen : sizeof(struct sockaddr_in6);
memcpy(&ipv6, r->ai_addr, len);
if (opal_net_samenetwork((struct sockaddr*)((struct sockaddr_in6*)&intf->if_addr),
(struct sockaddr*)&ipv6, intf->if_mask)) {
return intf->if_kernel_index;
}
}
@ -566,7 +550,7 @@ int opal_ifkindextoname(int if_kindex, char* if_name, int length)
bool
opal_ifislocal(const char *hostname)
{
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
char addrname[NI_MAXHOST]; /* should be larger than ADDRLEN, but I think
they really mean IFNAMESIZE */
#else

57
opal/util/net.c
Просмотреть файл

@ -12,6 +12,7 @@
* Copyright (c) 2007 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2009 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -82,31 +83,23 @@
/* this function doesn't depend on sockaddr_h */
bool opal_net_isaddr(const char *name)
{
unsigned int groups[8];
int i;
struct addrinfo hint, *res = NULL;
if (4 != sscanf(name, "%u.%u.%u.%u",
&groups[0], &groups[1], &groups[2], &groups[3])) {
/* this isn't an IPv4 address */
goto checkipv6;
}
/* initialize the hint */
memset(&hint, '\0', sizeof hint);
for (i=0; i < 4; i++) {
if (255 < groups[i]) {
return false;
}
}
return true;
/* indicate that we don't know the family */
hint.ai_family = PF_UNSPEC;
hint.ai_flags = AI_NUMERICHOST;
checkipv6:
/* TODO: deal with all the shorthand notations for IPv6! */
if (8 != sscanf(name, "%x:%x:%x:%x:%x:%x:%x:%x",
&groups[0], &groups[1], &groups[2], &groups[3],
&groups[4], &groups[5], &groups[6], &groups[7])) {
/* this isn't an IPv6 address */
if (0 != getaddrinfo(name, NULL, &hint, &res)) {
/* the input wasn't a recognizable address */
return false;
}
/* there are no value limits on the individual groups */
/* we don't care what family - all we care is that
* it is indeed an address
*/
freeaddrinfo(res);
return true;
}
@ -119,7 +112,7 @@ typedef struct private_ipv4_t {
static private_ipv4_t* private_ipv4 = NULL;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
static opal_tsd_key_t hostname_tsd_key;
@ -190,7 +183,7 @@ opal_net_init()
}
do_local_init:
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
return opal_tsd_key_create(&hostname_tsd_key, hostname_cleanup);
#else
return OPAL_SUCCESS;
@ -231,7 +224,7 @@ opal_net_islocalhost(const struct sockaddr *addr)
return false;
}
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
{
const struct sockaddr_in6 *inaddr = (struct sockaddr_in6*) addr;
@ -276,13 +269,13 @@ opal_net_samenetwork(const struct sockaddr *addr1,
uint32_t netmask = opal_net_prefix2netmask (prefixlen);
if((inaddr1->sin_addr.s_addr & netmask) ==
(inaddr2->sin_addr.s_addr & netmask)) {
(inaddr2->sin_addr.s_addr & netmask)) {
return true;
}
return false;
}
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
{
const struct sockaddr_in6 *inaddr1 = (struct sockaddr_in6*) addr1;
@ -295,7 +288,7 @@ opal_net_samenetwork(const struct sockaddr *addr1,
} else {
prefixlen = plen;
}
if (64 == prefixlen) {
if (64 == prefixlen) {
/* prefixlen is always /64, any other case would be routing.
Compare the first eight bytes (64 bits) and hope that
endianess is not an issue on any system as long as
@ -328,7 +321,7 @@ bool
opal_net_addr_isipv4public(const struct sockaddr *addr)
{
switch (addr->sa_family) {
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
return false;
#endif
@ -362,10 +355,11 @@ opal_net_addr_isipv4public(const struct sockaddr *addr)
char*
opal_net_get_hostname(const struct sockaddr *addr)
{
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
char *name = get_hostname_buffer();
int error;
socklen_t addrlen;
char *p;
if (NULL == name) {
opal_output(0, "opal_sockaddr2str: malloc() failed\n");
@ -407,7 +401,10 @@ opal_net_get_hostname(const struct sockaddr *addr)
free (name);
return NULL;
}
/* strip any trailing % data as it isn't pertinent */
if (NULL != (p = strrchr(name, '%'))) {
*p = '\0';
}
return name;
#else
return inet_ntoa(((struct sockaddr_in*) addr)->sin_addr);
@ -422,7 +419,7 @@ opal_net_get_port(const struct sockaddr *addr)
case AF_INET:
return ntohs(((struct sockaddr_in*) addr)->sin_port);
break;
#if OPAL_WANT_IPV6
#if OPAL_ENABLE_IPV6
case AF_INET6:
return ntohs(((struct sockaddr_in6*) addr)->sin6_port);
break;

4
orte/include/orte/constants.h
Просмотреть файл

@ -128,7 +128,9 @@ enum {
ORTE_ERR_COMM_DISABLED = (ORTE_ERR_BASE - 40),
ORTE_ERR_FAILED_TO_MAP = (ORTE_ERR_BASE - 41),
ORTE_ERR_SENSOR_LIMIT_EXCEEDED = (ORTE_ERR_BASE - 42),
ORTE_ERR_ALLOCATION_PENDING = (ORTE_ERR_BASE - 44)
ORTE_ERR_ALLOCATION_PENDING = (ORTE_ERR_BASE - 43),
ORTE_ERR_NO_PATH_TO_TARGET = (ORTE_ERR_BASE - 44),
ORTE_ERR_OP_IN_PROGRESS = (ORTE_ERR_BASE - 45)
};
#define ORTE_ERR_MAX (ORTE_ERR_BASE - 100)

30
orte/mca/dfs/app/dfs_app.c
Просмотреть файл

@ -100,18 +100,14 @@ static void recv_dfs(int status, orte_process_name_t* sender,
static int init(void)
{
int rc;
OBJ_CONSTRUCT(&requests, opal_list_t);
OBJ_CONSTRUCT(&active_files, opal_list_t);
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs,
NULL))) {
ORTE_ERROR_LOG(rc);
}
return rc;
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs,
NULL);
return ORTE_SUCCESS;
}
static int finalize(void)
@ -615,7 +611,7 @@ static void process_opens(int fd, short args, void *cbdata)
filename);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -721,7 +717,7 @@ static void process_close(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -843,7 +839,7 @@ static void process_sizes(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -997,7 +993,7 @@ static void process_seeks(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -1109,7 +1105,7 @@ static void process_reads(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -1176,7 +1172,7 @@ static void process_posts(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
goto error;
@ -1237,7 +1233,7 @@ static void process_getfm(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
goto error;

59
orte/mca/dfs/orted/dfs_orted.c
Просмотреть файл

@ -182,31 +182,24 @@ static void remote_seek(int fd, short args, void *cbdata);
static int init(void)
{
int rc;
int i;
worker_thread_t *wt;
OBJ_CONSTRUCT(&requests, opal_list_t);
OBJ_CONSTRUCT(&active_files, opal_list_t);
OBJ_CONSTRUCT(&file_maps, opal_list_t);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_CMD,
ORTE_RML_PERSISTENT,
recv_dfs_cmd,
NULL);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs_data,
NULL);
OBJ_CONSTRUCT(&worker_threads, opal_pointer_array_t);
opal_pointer_array_init(&worker_threads, 1, INT_MAX, 1);
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_CMD,
ORTE_RML_PERSISTENT,
recv_dfs_cmd,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs_data,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
opal_output_verbose(1, orte_dfs_base_framework.framework_output,
"%s starting %d worker threads",
@ -218,7 +211,7 @@ static int init(void)
opal_pointer_array_add(&worker_threads, wt);
}
return rc;
return ORTE_SUCCESS;
}
static int finalize(void)
@ -415,7 +408,7 @@ static void process_opens(int fd, short args, void *cbdata)
filename);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&node->daemon->name, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -515,7 +508,7 @@ static void process_close(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -620,7 +613,7 @@ static void process_sizes(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -768,7 +761,7 @@ static void process_seeks(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -878,7 +871,7 @@ static void process_reads(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -1375,7 +1368,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -1463,7 +1456,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -1572,7 +1565,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
(long)bytes_read,
ORTE_NAME_PRINT(sender));
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -1665,7 +1658,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
(long)bytes_read,
ORTE_NAME_PRINT(sender));
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -1720,7 +1713,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
return;
}
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -1889,7 +1882,7 @@ static void recv_dfs_cmd(int status, orte_process_name_t* sender,
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME), nmaps,
(int)answer->bytes_used, ORTE_NAME_PRINT(sender));
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -2144,7 +2137,7 @@ static void remote_open(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&req->trk->requestor, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -2185,7 +2178,7 @@ static void remote_size(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&req->trk->requestor, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -2249,7 +2242,7 @@ static void remote_seek(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&req->trk->requestor, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);
@ -2304,7 +2297,7 @@ static void remote_read(int fd, short args, void *cbdata)
(long)bytes_read,
ORTE_NAME_PRINT(&req->trk->requestor));
if (0 > (rc = orte_rml.send_buffer_nb(&req->trk->requestor, answer,
ORTE_RML_TAG_DFS_DATA, 0,
ORTE_RML_TAG_DFS_DATA,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(answer);

30
orte/mca/dfs/test/dfs_test.c
Просмотреть файл

@ -100,18 +100,14 @@ static void recv_dfs(int status, orte_process_name_t* sender,
static int init(void)
{
int rc;
OBJ_CONSTRUCT(&requests, opal_list_t);
OBJ_CONSTRUCT(&active_files, opal_list_t);
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs,
NULL))) {
ORTE_ERROR_LOG(rc);
}
return rc;
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DFS_DATA,
ORTE_RML_PERSISTENT,
recv_dfs,
NULL);
return ORTE_SUCCESS;
}
static int finalize(void)
@ -535,7 +531,7 @@ static void process_opens(int fd, short args, void *cbdata)
filename);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -635,7 +631,7 @@ static void process_close(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -735,7 +731,7 @@ static void process_sizes(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -845,7 +841,7 @@ static void process_seeks(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -941,7 +937,7 @@ static void process_reads(int fd, short args, void *cbdata)
trk->local_fd);
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(&trk->host_daemon, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -1008,7 +1004,7 @@ static void process_posts(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
goto error;
@ -1069,7 +1065,7 @@ static void process_getfm(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buffer,
ORTE_RML_TAG_DFS_CMD, 0,
ORTE_RML_TAG_DFS_CMD,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
goto error;

3
orte/mca/errmgr/Makefile.am
Просмотреть файл

@ -25,6 +25,9 @@ libmca_errmgr_la_SOURCES =
headers = errmgr.h
libmca_errmgr_la_SOURCES += $(headers)
# pkgdata setup
dist_pkgdata_DATA =
# Conditionally install the header files
if WANT_INSTALL_HEADERS
ortedir = $(includedir)/openmpi/$(subdir)

2
orte/mca/errmgr/base/Makefile.am
Просмотреть файл

@ -17,6 +17,8 @@
# $HEADER$
#
dist_pkgdata_DATA += base/help-errmgr-base.txt
headers += \
base/errmgr_private.h \
base/base.h

29
orte/mca/errmgr/base/help-errmgr-base.txt Обычный файл
Просмотреть файл

@ -0,0 +1,29 @@
# -*- text -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
#
[undeliverable-msg]
An attempt was made to send a message to a process whose
address is unknown:
Sender: %s
From node: %s
Recipient: %s
On node: %s
The message could not be delivered, and we are aborting.

41
orte/mca/errmgr/default_app/errmgr_default_app.c
Просмотреть файл

@ -31,9 +31,9 @@
#include "orte/util/error_strings.h"
#include "orte/util/name_fns.h"
#include "orte/util/show_help.h"
#include "orte/runtime/orte_globals.h"
#include "orte/mca/rml/rml.h"
#include "orte/mca/routed/routed.h"
#include "orte/mca/odls/odls_types.h"
#include "orte/mca/state/state.h"
@ -88,7 +88,9 @@ static int finalize(void)
static void proc_errors(int fd, short args, void *cbdata)
{
orte_state_caddy_t *caddy = (orte_state_caddy_t*)cbdata;
orte_ns_cmp_bitmask_t mask;
char *nodename;
orte_error_t err;
opal_pointer_array_t errors;
OPAL_OUTPUT_VERBOSE((1, orte_errmgr_base_framework.framework_output,
"%s errmgr:default_app: proc %s state %s",
@ -104,23 +106,28 @@ static void proc_errors(int fd, short args, void *cbdata)
return;
}
if (ORTE_PROC_STATE_COMM_FAILED == caddy->proc_state) {
mask = ORTE_NS_CMP_ALL;
/* if it is our own connection, ignore it */
if (OPAL_EQUAL == orte_util_compare_name_fields(mask, ORTE_PROC_MY_NAME, &caddy->name)) {
OBJ_RELEASE(caddy);
return;
}
/* see is this was a lifeline */
if (ORTE_SUCCESS != orte_routed.route_lost(&caddy->name)) {
/* order an exit */
ORTE_ERROR_LOG(ORTE_ERR_UNRECOVERABLE);
OBJ_RELEASE(caddy);
exit(1);
}
/* pass the error to the error_callbacks for processing */
OBJ_CONSTRUCT(&errors, opal_pointer_array_t);
opal_pointer_array_init(&errors, 1, INT_MAX, 1);
err.errcode = caddy->proc_state;
err.proc = caddy->name;
opal_pointer_array_add(&errors, &err);
if (ORTE_PROC_STATE_UNABLE_TO_SEND_MSG == caddy->proc_state) {
/* we can't send a message - print a message */
nodename = orte_get_proc_hostname(&caddy->name);
orte_show_help("help-errmgr-base",
"undeliverable-msg",
true, ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
orte_process_info.nodename,
ORTE_NAME_PRINT(&caddy->name),
(NULL == nodename) ? "Unknown" : nodename);
}
/* cleanup */
orte_errmgr_base_execute_error_callbacks(&errors);
OBJ_DESTRUCT(&errors);
OBJ_RELEASE(caddy);
}

21
orte/mca/errmgr/default_hnp/errmgr_default_hnp.c
Просмотреть файл

@ -248,14 +248,8 @@ static void proc_errors(int fd, short args, void *cbdata)
/* get the job object */
if (NULL == (jdata = orte_get_job_data_object(proc->jobid))) {
/* if the orteds are terminating, check job complete */
if (orte_orteds_term_ordered) {
opal_output(0, "TERM ORDERED - CHECKING COMPLETE");
goto cleanup;
} else {
ORTE_ERROR_LOG(ORTE_ERR_NOT_FOUND);
goto cleanup;
}
/* could be a race condition */
goto cleanup;
}
pptr = (orte_proc_t*)opal_pointer_array_get_item(jdata->procs, proc->vpid);
@ -279,6 +273,8 @@ static void proc_errors(int fd, short args, void *cbdata)
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
goto cleanup;
}
/* mark the daemon as gone */
pptr->alive = false;
/* if we have ordered orteds to terminate or abort
* is in progress, record it */
if (orte_orteds_term_ordered || orte_abnormal_term_ordered) {
@ -522,6 +518,15 @@ static void proc_errors(int fd, short args, void *cbdata)
default_hnp_abort(jdata);
break;
case ORTE_PROC_STATE_UNABLE_TO_SEND_MSG:
OPAL_OUTPUT_VERBOSE((5, orte_errmgr_base_framework.framework_output,
"%s errmgr:hnp: unable to send message to proc %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(proc)));
/* kill all jobs */
default_hnp_abort(jdata);
break;
default:
/* shouldn't get this, but terminate job if required */
OPAL_OUTPUT_VERBOSE((5, orte_errmgr_base_framework.framework_output,

42
orte/mca/errmgr/default_orted/errmgr_default_orted.c
Просмотреть файл

@ -173,7 +173,7 @@ static void job_errors(int fd, short args, void *cbdata)
/* kill all local procs */
killprocs(ORTE_JOBID_WILDCARD, ORTE_VPID_WILDCARD);
/* order termination */
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto cleanup;
break;
case ORTE_JOB_STATE_HEARTBEAT_FAILED:
@ -200,7 +200,7 @@ static void job_errors(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, alert,
ORTE_RML_TAG_PLM, 0,
ORTE_RML_TAG_PLM,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(alert);
@ -246,6 +246,20 @@ static void proc_errors(int fd, short args, void *cbdata)
/* if this was a failed comm, then see if it was to our
* lifeline
*/
if (ORTE_PROC_STATE_LIFELINE_LOST == state ||
ORTE_PROC_STATE_UNABLE_TO_SEND_MSG == state) {
OPAL_OUTPUT_VERBOSE((2, orte_errmgr_base_framework.framework_output,
"%s errmgr:orted lifeline lost - exiting",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
/* kill our children */
killprocs(ORTE_JOBID_WILDCARD, ORTE_VPID_WILDCARD);
/* terminate - our routed children will see
* us leave and automatically die
*/
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto cleanup;
}
if (ORTE_PROC_STATE_COMM_FAILED == state) {
/* if it is our own connection, ignore it */
if (OPAL_EQUAL == orte_util_compare_name_fields(ORTE_NS_CMP_ALL, ORTE_PROC_MY_NAME, proc)) {
@ -260,20 +274,6 @@ static void proc_errors(int fd, short args, void *cbdata)
"%s errmgr:default:orted daemon %s exited",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(proc)));
/* see if this was a lifeline */
if (ORTE_SUCCESS != orte_routed.route_lost(proc)) {
OPAL_OUTPUT_VERBOSE((2, orte_errmgr_base_framework.framework_output,
"%s errmgr:orted daemon %s was a lifeline - exiting",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(proc)));
/* kill our children */
killprocs(ORTE_JOBID_WILDCARD, ORTE_VPID_WILDCARD);
/* terminate - our routed children will see
* us leave and automatically die
*/
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto cleanup;
}
/* are any of my children still alive */
for (i=0; i < orte_local_children->size; i++) {
if (NULL != (child = (orte_proc_t*)opal_pointer_array_get_item(orte_local_children, i))) {
@ -288,7 +288,7 @@ static void proc_errors(int fd, short args, void *cbdata)
OPAL_OUTPUT_VERBOSE((2, orte_errmgr_base_framework.framework_output,
"%s errmgr:default:orted all routes gone - exiting",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
ORTE_TERMINATE(0);
ORTE_FORCED_TERMINATE(0);
} else {
OPAL_OUTPUT_VERBOSE((2, orte_errmgr_base_framework.framework_output,
"%s errmgr:default:orted not exiting, num_routes() == %d",
@ -402,7 +402,7 @@ static void proc_errors(int fd, short args, void *cbdata)
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, alert,
ORTE_RML_TAG_PLM, 0,
ORTE_RML_TAG_PLM,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(alert);
@ -466,7 +466,7 @@ static void proc_errors(int fd, short args, void *cbdata)
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, alert,
ORTE_RML_TAG_PLM, 0,
ORTE_RML_TAG_PLM,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
}
@ -523,7 +523,7 @@ static void proc_errors(int fd, short args, void *cbdata)
}
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, alert,
ORTE_RML_TAG_PLM, 0,
ORTE_RML_TAG_PLM,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
}
@ -573,7 +573,7 @@ static void proc_errors(int fd, short args, void *cbdata)
/* send it */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, alert,
ORTE_RML_TAG_PLM, 0,
ORTE_RML_TAG_PLM,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
}

37
orte/mca/ess/alps/ess_alps_module.c
Просмотреть файл

@ -118,23 +118,13 @@ static int rte_init(void)
/* as a tool, I don't need a nidmap - so just return now */
return ORTE_SUCCESS;
}
/* otherwise, I must be an application process - use
* the default procedure to finish my setup
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_setup())) {
ORTE_ERROR_LOG(ret);
error = "orte_ess_base_app_setup";
goto error;
}
/* setup the nidmap arrays */
if (ORTE_SUCCESS !=
(ret = orte_util_nidmap_init(orte_process_info.sync_buf))) {
ORTE_ERROR_LOG(ret);
error = "orte_util_nidmap_init";
goto error;
}
return ORTE_SUCCESS;
/* otherwise, I must be a direct-launched application process - if
* we were selected, then PMI isn't available and we cannot
* possibly run
*/
return ORTE_ERR_NOT_SUPPORTED;
error:
if (ORTE_ERR_SILENT != ret && !orte_report_silent_errors) {
@ -155,26 +145,15 @@ static int rte_finalize(void)
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
}
return ret;
} else if (ORTE_PROC_IS_TOOL) {
/* otherwise, if I am a tool proc, use that procedure */
if (ORTE_SUCCESS != (ret = orte_ess_base_tool_finalize())) {
ORTE_ERROR_LOG(ret);
}
/* as a tool, I didn't create a nidmap - so just return now */
return ret;
} else {
/* otherwise, I must be an application process
* use the default procedure to finish
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
}
}
/* deconstruct my nidmap and jobmap arrays */
orte_util_nidmap_finalize();
return ret;
return ORTE_SUCCESS;
}
static char *orte_ess_alps_jobid = NULL;

4
orte/mca/ess/base/ess_base_select.c
Просмотреть файл

@ -21,14 +21,10 @@
#include "orte_config.h"
#include "orte/constants.h"
#include "opal/class/opal_list.h"
#include "opal/mca/mca.h"
#include "opal/mca/base/base.h"
#include "opal/mca/base/mca_base_component_repository.h"
#include "orte/util/nidmap.h"
#include "orte/runtime/orte_globals.h"
#include "orte/mca/ess/base/base.h"
int

52
orte/mca/ess/base/ess_base_std_app.c
Просмотреть файл

@ -44,6 +44,7 @@
#include "orte/mca/errmgr/errmgr.h"
#include "orte/mca/dfs/base/base.h"
#include "orte/mca/grpcomm/base/base.h"
#include "orte/mca/oob/base/base.h"
#include "orte/mca/rml/rml.h"
#include "orte/mca/odls/odls_types.h"
#include "orte/mca/plm/plm.h"
@ -109,6 +110,19 @@ int orte_ess_base_app_setup(void)
}
/* Setup the communication infrastructure */
/*
* OOB Layer
*/
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&orte_oob_base_framework, 0))) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_open";
goto error;
}
if (ORTE_SUCCESS != (ret = orte_oob_base_select())) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_select";
goto error;
}
/* Runtime Messaging Layer */
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&orte_rml_base_framework, 0))) {
@ -274,12 +288,10 @@ int orte_ess_base_app_setup(void)
}
/* if we are an ORTE app - and not an MPI app - then
* we need to barrier here. MPI_Init has its own barrier,
* so we don't need to do two of them. However, if we
* don't do a barrier at all, then one process could
* finalize before another one called orte_init. This
* causes ORTE to believe that the proc abnormally
* terminated
* we need to exchange our connection info here.
* MPI_Init has its own modex, so we don't need to do
* two of them. However, if we don't do a modex at all,
* then processes have no way to communicate
*
* NOTE: only do this when the process originally launches.
* Cannot do this on a restart as the rest of the processes
@ -288,12 +300,13 @@ int orte_ess_base_app_setup(void)
if (ORTE_PROC_IS_NON_MPI && !orte_do_not_barrier) {
orte_grpcomm_collective_t coll;
OBJ_CONSTRUCT(&coll, orte_grpcomm_collective_t);
coll.id = orte_process_info.peer_init_barrier;
if (ORTE_SUCCESS != (ret = orte_grpcomm.barrier(&coll))) {
coll.id = orte_process_info.peer_modex;
if (ORTE_SUCCESS != (ret = orte_grpcomm.modex(&coll))) {
ORTE_ERROR_LOG(ret);
error = "orte barrier";
error = "orte modex";
goto error;
}
coll.active = true;
ORTE_WAIT_FOR_COMPLETION(coll.active);
OBJ_DESTRUCT(&coll);
}
@ -311,7 +324,7 @@ error:
int orte_ess_base_app_finalize(void)
{
orte_cr_finalize();
#if OPAL_ENABLE_FT_CR == 1
(void) mca_base_framework_close(&orte_snapc_base_framework);
(void) mca_base_framework_close(&orte_sstore_base_framework);
@ -327,7 +340,8 @@ int orte_ess_base_app_finalize(void)
(void) mca_base_framework_close(&orte_dfs_base_framework);
(void) mca_base_framework_close(&orte_routed_base_framework);
(void) mca_base_framework_close(&orte_rml_base_framework);
(void) mca_base_framework_close(&orte_oob_base_framework);
orte_session_dir_finalize(ORTE_PROC_MY_NAME);
return ORTE_SUCCESS;
@ -359,21 +373,21 @@ int orte_ess_base_app_finalize(void)
* to prevent the abort file from being created. This allows the
* session directory tree to cleanly be eliminated.
*/
static bool sync_waiting = false;
static void report_sync(int status, orte_process_name_t* sender,
opal_buffer_t *buffer,
orte_rml_tag_t tag, void *cbdata)
{
bool *sync_waiting = (bool*)cbdata;
/* flag as complete */
sync_waiting = false;
*sync_waiting = false;
}
void orte_ess_base_app_abort(int status, bool report)
{
orte_daemon_cmd_flag_t cmd=ORTE_DAEMON_ABORT_CALLED;
opal_buffer_t *buf;
bool sync_waiting = true;
/* Exit - do NOT do a normal finalize as this will very likely
* hang the process. We are aborting due to an abnormal condition
* that precludes normal cleanup
@ -390,7 +404,7 @@ void orte_ess_base_app_abort(int status, bool report)
if (report) {
buf = OBJ_NEW(opal_buffer_t);
opal_dss.pack(buf, &cmd, 1, ORTE_DAEMON_CMD);
orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buf, ORTE_RML_TAG_DAEMON, 0, orte_rml_send_callback, NULL);
orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buf, ORTE_RML_TAG_DAEMON, orte_rml_send_callback, NULL);
OPAL_OUTPUT_VERBOSE((5, orte_debug_output,
"%s orte_ess_app_abort: sent abort msg to %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
@ -400,10 +414,8 @@ void orte_ess_base_app_abort(int status, bool report)
* process exiting
*/
sync_waiting = true;
if (ORTE_SUCCESS != orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_ABORT,
ORTE_RML_NON_PERSISTENT, report_sync, NULL)) {
exit(status);
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_ABORT,
ORTE_RML_NON_PERSISTENT, report_sync, &sync_waiting);
ORTE_WAIT_FOR_COMPLETION(sync_waiting);
}

32
orte/mca/ess/base/ess_base_std_orted.c
Просмотреть файл

@ -14,6 +14,7 @@
* Copyright (c) 2011 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2011-2013 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -37,7 +38,6 @@
#include "opal/mca/event/event.h"
#include "opal/runtime/opal.h"
#include "opal/runtime/opal_cr.h"
#include "opal/mca/db/base/base.h"
#include "opal/mca/hwloc/base/base.h"
#include "opal/mca/pstat/base/base.h"
#include "opal/util/os_path.h"
@ -45,6 +45,7 @@
#include "orte/mca/rml/base/base.h"
#include "orte/mca/routed/base/base.h"
#include "orte/mca/routed/routed.h"
#include "orte/mca/oob/base/base.h"
#include "orte/mca/dfs/base/base.h"
#include "orte/mca/grpcomm/grpcomm.h"
#include "orte/mca/grpcomm/base/base.h"
@ -233,8 +234,16 @@ int orte_ess_base_orted_setup(char **hosts)
}
/* Setup the communication infrastructure */
/* Runtime Messaging Layer - this opens/selects the OOB as well */
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&orte_oob_base_framework, 0))) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_open";
goto error;
}
if (ORTE_SUCCESS != (ret = orte_oob_base_select())) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_select";
goto error;
}
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&orte_rml_base_framework, 0))) {
ORTE_ERROR_LOG(ret);
error = "orte_rml_base_open";
@ -265,18 +274,6 @@ int orte_ess_base_orted_setup(char **hosts)
goto error;
}
/* database */
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&opal_db_base_framework, 0))) {
ORTE_ERROR_LOG(ret);
error = "orte_db_base_open";
goto error;
}
if (ORTE_SUCCESS != (ret = opal_db_base_select())) {
ORTE_ERROR_LOG(ret);
error = "orte_db_base_select";
goto error;
}
/*
* Group communications
*/
@ -648,6 +645,7 @@ int orte_ess_base_orted_finalize(void)
(void) mca_base_framework_close(&orte_odls_base_framework);
(void) mca_base_framework_close(&orte_routed_base_framework);
(void) mca_base_framework_close(&orte_rml_base_framework);
(void) mca_base_framework_close(&orte_oob_base_framework);
(void) mca_base_framework_close(&orte_state_base_framework);
/* cleanup any lingering session directories */
@ -671,9 +669,7 @@ static void shutdown_signal(int fd, short flags, void *arg)
*/
static void epipe_signal_callback(int fd, short flags, void *arg)
{
/* for now, we just announce and ignore them */
opal_output(0, "%s reports a SIGPIPE error on fd %d",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME), fd);
/* for now, we just ignore them */
return;
}

25
orte/mca/ess/env/ess_env_module.c поставляемый
Просмотреть файл

@ -178,12 +178,13 @@ error:
static int rte_finalize(void)
{
int ret;
/* if I am a daemon, finalize using the default procedure */
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
}
return ret;
} else if (ORTE_PROC_IS_TOOL) {
/* otherwise, if I am a tool proc, use that procedure */
if (ORTE_SUCCESS != (ret = orte_ess_base_tool_finalize())) {
@ -191,19 +192,19 @@ static int rte_finalize(void)
}
/* as a tool, I didn't create a nidmap - so just return now */
return ret;
} else {
/* otherwise, I must be an application process
* use the default procedure to finish
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
}
}
/* otherwise, I must be an application process
* use the default procedure to finish
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
}
/* deconstruct the nidmap and jobmap arrays */
orte_util_nidmap_finalize();
return ret;
return ORTE_SUCCESS;
}
static int env_set_name(void)
@ -328,6 +329,7 @@ static int rte_ft_event(int state)
exit_status = ret;
goto cleanup;
}
coll.active = true;
ORTE_WAIT_FOR_COMPLETION(coll.active);
if( orte_cr_flush_restart_files ) {
@ -453,6 +455,7 @@ static int rte_ft_event(int state)
exit_status = ret;
goto cleanup;
}
coll.active = true;
ORTE_WAIT_FOR_COMPLETION(coll.active);
if( orte_cr_flush_restart_files ) {

1
orte/mca/ess/ess.h
Просмотреть файл

@ -34,6 +34,7 @@
#include "opal/mca/hwloc/base/base.h"
#include "orte/util/proc_info.h"
#include "orte/runtime/runtime.h"
BEGIN_C_DECLS

234
orte/mca/ess/hnp/ess_hnp_module.c
Просмотреть файл

@ -13,6 +13,7 @@
* Copyright (c) 2011-2012 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2011-2013 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -48,6 +49,7 @@
#include "opal/mca/pstat/base/base.h"
#include "opal/mca/hwloc/base/base.h"
#include "orte/mca/oob/base/base.h"
#include "orte/mca/rml/base/base.h"
#include "orte/mca/rml/rml_types.h"
#include "orte/mca/routed/base/base.h"
@ -290,6 +292,20 @@ static int rte_init(void)
/* Setup the communication infrastructure */
/*
* OOB Layer
*/
if (ORTE_SUCCESS != (ret = mca_base_framework_open(&orte_oob_base_framework, 0))) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_open";
goto error;
}
if (ORTE_SUCCESS != (ret = orte_oob_base_select())) {
ORTE_ERROR_LOG(ret);
error = "orte_oob_base_select";
goto error;
}
/*
* Runtime Messaging Layer
*/
@ -310,6 +326,103 @@ static int rte_init(void)
goto error;
}
/* setup the global job and node arrays */
orte_job_data = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_job_data,
1,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
1))) {
ORTE_ERROR_LOG(ret);
error = "setup job array";
goto error;
}
orte_node_pool = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_node_pool,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE))) {
ORTE_ERROR_LOG(ret);
error = "setup node array";
goto error;
}
orte_node_topologies = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_node_topologies,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE))) {
ORTE_ERROR_LOG(ret);
error = "setup node topologies array";
goto error;
}
/* init the nidmap - just so we register that verbosity */
orte_util_nidmap_init(NULL);
/* Setup the job data object for the daemons */
/* create and store the job data object */
jdata = OBJ_NEW(orte_job_t);
jdata->jobid = ORTE_PROC_MY_NAME->jobid;
opal_pointer_array_set_item(orte_job_data, 0, jdata);
/* mark that the daemons have reported as we are the
* only ones in the system right now, and we definitely
* are running!
*/
jdata->state = ORTE_JOB_STATE_DAEMONS_REPORTED;
/* every job requires at least one app */
app = OBJ_NEW(orte_app_context_t);
opal_pointer_array_set_item(jdata->apps, 0, app);
jdata->num_apps++;
/* create and store a node object where we are */
node = OBJ_NEW(orte_node_t);
node->name = strdup(orte_process_info.nodename);
node->index = opal_pointer_array_set_item(orte_node_pool, 0, node);
#if OPAL_HAVE_HWLOC
/* point our topology to the one detected locally */
node->topology = opal_hwloc_topology;
/* add it to the array of known topologies */
opal_pointer_array_add(orte_node_topologies, opal_hwloc_topology);
#endif
/* create and store a proc object for us */
proc = OBJ_NEW(orte_proc_t);
proc->name.jobid = ORTE_PROC_MY_NAME->jobid;
proc->name.vpid = ORTE_PROC_MY_NAME->vpid;
proc->pid = orte_process_info.pid;
proc->rml_uri = orte_rml.get_contact_info();
proc->state = ORTE_PROC_STATE_RUNNING;
OBJ_RETAIN(node); /* keep accounting straight */
proc->node = node;
proc->nodename = node->name;
opal_pointer_array_set_item(jdata->procs, proc->name.vpid, proc);
/* record that the daemon (i.e., us) is on this node
* NOTE: we do not add the proc object to the node's
* proc array because we are not an application proc.
* Instead, we record it in the daemon field of the
* node object
*/
OBJ_RETAIN(proc); /* keep accounting straight */
node->daemon = proc;
node->daemon_launched = true;
node->state = ORTE_NODE_STATE_UP;
/* if we are to retain aliases, get ours */
if (orte_retain_aliases) {
opal_ifgetaliases(&node->alias);
/* add our own local name to it */
opal_argv_append_nosize(&node->alias, orte_process_info.nodename);
}
/* record that the daemon job is running */
jdata->num_procs = 1;
jdata->state = ORTE_JOB_STATE_RUNNING;
/* obviously, we have "reported" */
jdata->num_reported = 1;
/*
* Routed system
*/
@ -410,18 +523,14 @@ static int rte_init(void)
/* we are an hnp, so update the contact info field for later use */
orte_process_info.my_hnp_uri = orte_rml.get_contact_info();
proc->rml_uri = strdup(orte_process_info.my_hnp_uri);
/* we are also officially a daemon, so better update that field too */
orte_process_info.my_daemon_uri = orte_rml.get_contact_info();
orte_process_info.my_daemon_uri = strdup(orte_process_info.my_hnp_uri);
/* setup the orte_show_help system to recv remote output */
ret = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_SHOW_HELP,
ORTE_RML_PERSISTENT, orte_show_help_recv, NULL);
if (ret != ORTE_SUCCESS && ret != ORTE_ERR_NOT_IMPLEMENTED) {
ORTE_ERROR_LOG(ret);
error = "setup receive for orte_show_help";
goto error;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD, ORTE_RML_TAG_SHOW_HELP,
ORTE_RML_PERSISTENT, orte_show_help_recv, NULL);
/* setup my session directory */
if (orte_create_session_dirs) {
@ -469,103 +578,6 @@ static int rte_init(void)
free(contact_path);
}
/* init the nidmap - just so we register that verbosity */
orte_util_nidmap_init(NULL);
/* setup the global job and node arrays */
orte_job_data = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_job_data,
1,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
1))) {
ORTE_ERROR_LOG(ret);
error = "setup job array";
goto error;
}
orte_node_pool = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_node_pool,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE))) {
ORTE_ERROR_LOG(ret);
error = "setup node array";
goto error;
}
orte_node_topologies = OBJ_NEW(opal_pointer_array_t);
if (ORTE_SUCCESS != (ret = opal_pointer_array_init(orte_node_topologies,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
ORTE_GLOBAL_ARRAY_MAX_SIZE,
ORTE_GLOBAL_ARRAY_BLOCK_SIZE))) {
ORTE_ERROR_LOG(ret);
error = "setup node topologies array";
goto error;
}
/* Setup the job data object for the daemons */
/* create and store the job data object */
jdata = OBJ_NEW(orte_job_t);
jdata->jobid = ORTE_PROC_MY_NAME->jobid;
opal_pointer_array_set_item(orte_job_data, 0, jdata);
/* mark that the daemons have reported as we are the
* only ones in the system right now, and we definitely
* are running!
*/
jdata->state = ORTE_JOB_STATE_DAEMONS_REPORTED;
/* every job requires at least one app */
app = OBJ_NEW(orte_app_context_t);
opal_pointer_array_set_item(jdata->apps, 0, app);
jdata->num_apps++;
/* create and store a node object where we are */
node = OBJ_NEW(orte_node_t);
node->name = strdup(orte_process_info.nodename);
node->index = opal_pointer_array_set_item(orte_node_pool, 0, node);
#if OPAL_HAVE_HWLOC
/* point our topology to the one detected locally */
node->topology = opal_hwloc_topology;
/* add it to the array of known topologies */
opal_pointer_array_add(orte_node_topologies, opal_hwloc_topology);
#endif
/* create and store a proc object for us */
proc = OBJ_NEW(orte_proc_t);
proc->name.jobid = ORTE_PROC_MY_NAME->jobid;
proc->name.vpid = ORTE_PROC_MY_NAME->vpid;
proc->pid = orte_process_info.pid;
proc->rml_uri = orte_rml.get_contact_info();
proc->state = ORTE_PROC_STATE_RUNNING;
OBJ_RETAIN(node); /* keep accounting straight */
proc->node = node;
proc->nodename = node->name;
opal_pointer_array_set_item(jdata->procs, proc->name.vpid, proc);
/* record that the daemon (i.e., us) is on this node
* NOTE: we do not add the proc object to the node's
* proc array because we are not an application proc.
* Instead, we record it in the daemon field of the
* node object
*/
OBJ_RETAIN(proc); /* keep accounting straight */
node->daemon = proc;
node->daemon_launched = true;
node->state = ORTE_NODE_STATE_UP;
/* if we are to retain aliases, get ours */
if (orte_retain_aliases) {
opal_ifgetaliases(&node->alias);
/* add our own local name to it */
opal_argv_append_nosize(&node->alias, orte_process_info.nodename);
}
/* record that the daemon job is running */
jdata->num_procs = 1;
jdata->state = ORTE_JOB_STATE_RUNNING;
/* obviously, we have "reported" */
jdata->num_reported = 1;
/* setup the routed info - the selected routed component
* will know what to do.
*/
@ -749,7 +761,6 @@ static int rte_finalize(void)
(void) mca_base_framework_close(&orte_plm_base_framework);
(void) mca_base_framework_close(&orte_errmgr_base_framework);
(void) mca_base_framework_close(&orte_state_base_framework);
(void) mca_base_framework_close(&orte_rml_base_framework);
/* cleanup the pstat stuff */
(void) mca_base_framework_close(&opal_pstat_base_framework);
@ -763,6 +774,10 @@ static int rte_finalize(void)
free(contact_path);
}
/* shutdown the messaging frameworks */
(void) mca_base_framework_close(&orte_rml_base_framework);
(void) mca_base_framework_close(&orte_oob_base_framework);
/* ensure we scrub the session directory tree */
orte_session_dir_cleanup(ORTE_JOBID_WILDCARD);
@ -774,8 +789,7 @@ static int rte_finalize(void)
fclose(orte_xml_fp);
}
}
return ORTE_SUCCESS;
return ORTE_SUCCESS;
}
static void rte_abort(int status, bool report)
@ -848,7 +862,7 @@ static void clean_abort(int fd, short flags, void *arg)
(which is a Bad Thing), so we can't call it directly.
Instead, we have to exit this handler and setup to call
job_completed() after this. */
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
}
static struct timeval current, last={0,0};

9
orte/mca/ess/lsf/ess_lsf_module.c
Просмотреть файл

@ -146,6 +146,7 @@ static int rte_finalize(void)
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
} else if (ORTE_PROC_IS_TOOL) {
/* otherwise, if I am a tool proc, use that procedure */
@ -160,14 +161,14 @@ static int rte_finalize(void)
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
}
/* deconstruct my nidmap and jobmap arrays */
orte_util_nidmap_finalize();
return ret;
return ORTE_SUCCESS;;
}
static int lsf_set_name(void)

17
orte/mca/ess/pmi/ess_pmi_module.c
Просмотреть файл

@ -424,25 +424,27 @@ static int rte_init(void)
static int rte_finalize(void)
{
int ret = ORTE_SUCCESS;
int ret;
if (app_init_complete) {
/* if I am a daemon, finalize using the default procedure */
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
} else {
/* use the default app procedure to finish */
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
}
/* remove the envars that we pushed into environ
* so we leave that structure intact
*/
unsetenv("OMPI_MCA_grpcomm");
unsetenv("OMPI_MCA_routed");
unsetenv("OMPI_MCA_orte_precondition_transports");
/* use the default app procedure to finish */
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
}
}
@ -458,8 +460,7 @@ static int rte_finalize(void)
opal_hwloc_topology = NULL;
}
#endif
return ret;
return ORTE_SUCCESS;
}
static void rte_abort(int error_code, bool report)

3
orte/mca/ess/singleton/ess_singleton_module.c
Просмотреть файл

@ -250,8 +250,7 @@ static int rte_finalize(void)
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
}
return ret;
return ret;
}

6
orte/mca/ess/slurm/ess_slurm_module.c
Просмотреть файл

@ -133,6 +133,7 @@ static int rte_finalize(void)
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
} else if (ORTE_PROC_IS_TOOL) {
/* otherwise, if I am a tool proc, use that procedure */
@ -147,13 +148,14 @@ static int rte_finalize(void)
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
}
/* deconstruct my nidmap and jobmap arrays */
orte_util_nidmap_finalize();
return ret;
return ORTE_SUCCESS;
}
static int slurm_set_name(void)

6
orte/mca/ess/tm/ess_tm_module.c
Просмотреть файл

@ -148,6 +148,7 @@ static int rte_finalize(void)
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (ret = orte_ess_base_orted_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
} else if (ORTE_PROC_IS_TOOL) {
/* otherwise, if I am a tool proc, use that procedure */
@ -162,13 +163,14 @@ static int rte_finalize(void)
*/
if (ORTE_SUCCESS != (ret = orte_ess_base_app_finalize())) {
ORTE_ERROR_LOG(ret);
return ret;
}
}
/* deconstruct my nidmap and jobmap arrays */
orte_util_nidmap_finalize();
return ret;
return ORTE_SUCCESS;
}
static int tm_set_name(void)

2
orte/mca/filem/base/filem_base_fns.c
Просмотреть файл

@ -34,8 +34,6 @@
#include "opal/mca/mca.h"
#include "opal/mca/base/base.h"
#include "orte/mca/rml/rml.h"
#include "orte/mca/rml/rml_types.h"
#include "orte/mca/errmgr/errmgr.h"
#include "orte/runtime/orte_globals.h"
#include "orte/util/proc_info.h"

101
orte/mca/filem/base/filem_base_receive.c
Просмотреть файл

@ -10,7 +10,7 @@
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2011 Los Alamos National Security, LLC. All rights
* Copyright (c) 2011-2012 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
@ -70,8 +70,6 @@ static bool recv_issued=false;
int orte_filem_base_comm_start(void)
{
int rc;
/* Only active in HNP and daemons */
if( !ORTE_PROC_IS_HNP && !ORTE_PROC_IS_DAEMON ) {
return ORTE_SUCCESS;
@ -84,24 +82,20 @@ int orte_filem_base_comm_start(void)
"%s filem:base: Receive: Start command recv",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE,
ORTE_RML_PERSISTENT,
orte_filem_base_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE,
ORTE_RML_PERSISTENT,
orte_filem_base_recv,
NULL);
recv_issued = true;
return rc;
return ORTE_SUCCESS;
}
int orte_filem_base_comm_stop(void)
{
int rc;
/* Only active in HNP and daemons */
if( !ORTE_PROC_IS_HNP && !ORTE_PROC_IS_DAEMON ) {
return ORTE_SUCCESS;
@ -114,12 +108,10 @@ int orte_filem_base_comm_stop(void)
"%s filem:base:receive stop comm",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
if (ORTE_SUCCESS != (rc = orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_FILEM_BASE))) {
ORTE_ERROR_LOG(rc);
}
orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_FILEM_BASE);
recv_issued = false;
return rc;
return ORTE_SUCCESS;
}
@ -171,23 +163,21 @@ void orte_filem_base_recv(int status, orte_process_name_t* sender,
static void filem_base_process_get_proc_node_name_cmd(orte_process_name_t* sender,
opal_buffer_t* buffer)
{
opal_buffer_t answer;
opal_buffer_t *answer;
orte_std_cntr_t count;
orte_job_t *jdata = NULL;
orte_proc_t **procs = NULL;
orte_proc_t *proc = NULL;
orte_process_name_t name;
int rc;
OBJ_CONSTRUCT(&answer, opal_buffer_t);
/*
* Unpack the data
*/
count = 1;
if (ORTE_SUCCESS != (rc = opal_dss.unpack(buffer, &name, &count, ORTE_NAME))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto CLEANUP;
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
return;
}
/*
@ -196,34 +186,36 @@ static void filem_base_process_get_proc_node_name_cmd(orte_process_name_t* sende
/* get the job data object for this proc */
if (NULL == (jdata = orte_get_job_data_object(name.jobid))) {
ORTE_ERROR_LOG(ORTE_ERR_NOT_FOUND);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto CLEANUP;
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
return;
}
/* get the proc object for it */
procs = (orte_proc_t**)jdata->procs->addr;
if (NULL == procs[name.vpid] || NULL == procs[name.vpid]->node) {
proc = (orte_proc_t*)opal_pointer_array_get_item(jdata->procs, name.vpid);
if (NULL == proc || NULL == proc->node) {
ORTE_ERROR_LOG(ORTE_ERR_NOT_FOUND);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto CLEANUP;
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
return;
}
/*
* Send back the answer
*/
if (ORTE_SUCCESS != (rc = opal_dss.pack(&answer, &(procs[name.vpid]->node->name), 1, OPAL_STRING))) {
answer = OBJ_NEW(opal_buffer_t);
if (ORTE_SUCCESS != (rc = opal_dss.pack(answer, &(proc->node->name), 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto CLEANUP;
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
OBJ_RELEASE(answer);
return;
}
if (0 > (rc = orte_rml.send_buffer(sender, &answer, ORTE_RML_TAG_FILEM_BASE_RESP, 0))) {
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_FILEM_BASE_RESP,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
OBJ_RELEASE(answer);
return;
}
CLEANUP:
OBJ_DESTRUCT(&answer);
}
/*
@ -235,7 +227,7 @@ static void filem_base_process_get_proc_node_name_cmd(orte_process_name_t* sende
static void filem_base_process_get_remote_path_cmd(orte_process_name_t* sender,
opal_buffer_t* buffer)
{
opal_buffer_t answer;
opal_buffer_t *answer;
orte_std_cntr_t count;
char *filename = NULL;
char *tmp_name = NULL;
@ -244,22 +236,17 @@ static void filem_base_process_get_remote_path_cmd(orte_process_name_t* sender,
struct stat file_status;
int rc;
/*
* Unpack the data
*/
OBJ_CONSTRUCT(&answer, opal_buffer_t);
count = 1;
if (ORTE_SUCCESS != (rc = opal_dss.unpack(buffer, &filename, &count, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto CLEANUP;
}
/*
* Determine the absolute path of the file
*/
if(filename[0] != '/') { /* if it is not an absolute path already */
if (filename[0] != '/') { /* if it is not an absolute path already */
getcwd(cwd, sizeof(cwd));
asprintf(&tmp_name, "%s/%s", cwd, filename);
}
@ -277,7 +264,7 @@ static void filem_base_process_get_remote_path_cmd(orte_process_name_t* sender,
* Determine if the file/dir exists at that absolute path
* Determine if the file/dir is a file or a directory
*/
if(0 != (rc = stat(tmp_name, &file_status) ) ){
if (0 != (rc = stat(tmp_name, &file_status) ) ){
file_type = ORTE_FILEM_TYPE_UNKNOWN;
}
else {
@ -297,20 +284,26 @@ static void filem_base_process_get_remote_path_cmd(orte_process_name_t* sender,
* - ORTE_FILEM_TYPE_DIR = Directory
* - ORTE_FILEM_TYPE_UNKNOWN = Could not be determined, or does not exist
*/
if (ORTE_SUCCESS != (rc = opal_dss.pack(&answer, &tmp_name, 1, OPAL_STRING))) {
answer = OBJ_NEW(opal_buffer_t);
if (ORTE_SUCCESS != (rc = opal_dss.pack(answer, &tmp_name, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
OBJ_RELEASE(answer);
goto CLEANUP;
}
if (ORTE_SUCCESS != (rc = opal_dss.pack(&answer, &file_type, 1, OPAL_INT))) {
if (ORTE_SUCCESS != (rc = opal_dss.pack(answer, &file_type, 1, OPAL_INT))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
OBJ_RELEASE(answer);
goto CLEANUP;
}
if (0 > (rc = orte_rml.send_buffer(sender, &answer, ORTE_RML_TAG_FILEM_BASE_RESP, 0))) {
if (0 > (rc = orte_rml.send_buffer_nb(sender, answer,
ORTE_RML_TAG_FILEM_BASE_RESP,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
OBJ_RELEASE(answer);
}
CLEANUP:
@ -322,6 +315,4 @@ static void filem_base_process_get_remote_path_cmd(orte_process_name_t* sender,
free(tmp_name);
tmp_name = NULL;
}
OBJ_DESTRUCT(&answer);
}

32
orte/mca/filem/raw/filem_raw_module.c
Просмотреть файл

@ -104,35 +104,27 @@ static void write_handler(int fd, short event, void *cbdata);
static int raw_init(void)
{
int rc;
OBJ_CONSTRUCT(&incoming_files, opal_list_t);
/* start a recv to catch any files sent to me */
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE,
ORTE_RML_PERSISTENT,
recv_files,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE,
ORTE_RML_PERSISTENT,
recv_files,
NULL);
/* if I'm the HNP, start a recv to catch acks sent to me */
if (ORTE_PROC_IS_HNP) {
OBJ_CONSTRUCT(&outbound_files, opal_list_t);
OBJ_CONSTRUCT(&positioned_files, opal_list_t);
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE_RESP,
ORTE_RML_PERSISTENT,
recv_ack,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_FILEM_BASE_RESP,
ORTE_RML_PERSISTENT,
recv_ack,
NULL);
}
return rc;
return ORTE_SUCCESS;
}
static int raw_finalize(void)
@ -911,7 +903,7 @@ static void send_complete(char *file, int status)
return;
}
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf,
ORTE_RML_TAG_FILEM_BASE_RESP, 0,
ORTE_RML_TAG_FILEM_BASE_RESP,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);

40
orte/mca/grpcomm/bad/grpcomm_bad_module.c
Просмотреть файл

@ -120,10 +120,10 @@ static int xcast(orte_jobid_t job,
ORTE_ERROR_LOG(rc);
goto CLEANUP;
}
/* send it to the HNP (could be myself) for relay */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf, ORTE_RML_TAG_XCAST,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
goto CLEANUP;
@ -165,7 +165,7 @@ static void process_barrier(int fd, short args, void *cbdata)
/* send the buffer to my daemon */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buf, ORTE_RML_TAG_COLLECTIVE,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
opal_list_remove_item(&orte_grpcomm_base.active_colls, &coll->super);
@ -183,18 +183,6 @@ static int bad_barrier(orte_grpcomm_collective_t *coll)
"%s grpcomm:bad entering barrier",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
/* if I am alone, just execute the callback */
if (1 == orte_process_info.num_procs) {
coll->active = false;
if (NULL != coll->cbfunc) {
coll->cbfunc(NULL, coll->cbdata);
}
return ORTE_SUCCESS;
}
/* mark the collective as active */
coll->active = true;
/* push it into the event library for processing as
* we will be accessing global lists
*/
@ -237,7 +225,7 @@ static void process_allgather(int fd, short args, void *cbdata)
(int)gather->id));
/* send to our daemon */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_DAEMON, buf,
ORTE_RML_TAG_COLLECTIVE, 0,
ORTE_RML_TAG_COLLECTIVE,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
@ -261,7 +249,7 @@ static void process_allgather(int fd, short args, void *cbdata)
(int)gather->id,
ORTE_NAME_PRINT(&nm->name)));
if (0 > (rc = orte_rml.send_buffer_nb(&nm->name, buf,
ORTE_RML_TAG_COLLECTIVE, 0,
ORTE_RML_TAG_COLLECTIVE,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
@ -283,21 +271,6 @@ static int bad_allgather(orte_grpcomm_collective_t *gather)
"%s grpcomm:bad entering allgather",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
/* if I am alone and nobody else is participating, then
* nothing really to do
*/
if (1 == orte_process_info.num_procs &&
0 == opal_list_get_size(&gather->participants)) {
gather->active = false;
if (NULL != gather->cbfunc) {
gather->cbfunc(&gather->buffer, gather->cbdata);
}
return ORTE_SUCCESS;
}
/* mark the collective as active */
gather->active = true;
/* push it into the event library for processing as
* we will be accessing global lists
*/
@ -307,9 +280,6 @@ static int bad_allgather(orte_grpcomm_collective_t *gather)
static int bad_modex(orte_grpcomm_collective_t *modex)
{
/* mark the collective as active */
modex->active = true;
/* we need to get this into the event library
* to avoid race conditions with modex data arriving
* from other sources via the RML

31
orte/mca/grpcomm/base/grpcomm_base_modex.c
Просмотреть файл

@ -40,6 +40,7 @@
#include "orte/util/proc_info.h"
#include "orte/mca/errmgr/errmgr.h"
#include "orte/mca/ess/ess.h"
#include "orte/mca/rml/rml.h"
#include "orte/runtime/orte_globals.h"
#include "orte/util/name_fns.h"
#include "orte/util/nidmap.h"
@ -71,6 +72,7 @@ void orte_grpcomm_base_modex(int fd, short args, void *cbdata)
opal_list_item_t *item;
bool found;
orte_grpcomm_collective_t *cptr;
char *rml_uri;
OPAL_OUTPUT_VERBOSE((1, orte_grpcomm_base_framework.framework_output,
"%s grpcomm:base:modex: performing modex",
@ -87,6 +89,16 @@ void orte_grpcomm_base_modex(int fd, short args, void *cbdata)
goto cleanup;
}
if (ORTE_PROC_IS_NON_MPI) {
/* add my RML contact info */
rml_uri = orte_rml.get_contact_info();
if (ORTE_SUCCESS != (rc = opal_dss.pack(&modex->buffer, &rml_uri, 1, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
goto cleanup;
}
free(rml_uri);
}
/* add a wildcard name to the participants so the daemon knows
* the jobid that is involved in this collective
*/
@ -372,8 +384,6 @@ void orte_grpcomm_base_store_peer_modex(opal_buffer_t *rbuf, void *cbdata)
}
cleanup:
/* flag the collective as complete */
modex->active = false;
/* cleanup the list, but don't release the
* collective object as it was passed into us
*/
@ -389,6 +399,8 @@ void orte_grpcomm_base_store_peer_modex(opal_buffer_t *rbuf, void *cbdata)
"%s store:peer:modex NO MODEX RELEASE CBFUNC",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
}
/* flag the collective as complete */
modex->active = false;
}
void orte_grpcomm_base_store_modex(opal_buffer_t *rbuf, void *cbdata)
@ -397,6 +409,7 @@ void orte_grpcomm_base_store_modex(opal_buffer_t *rbuf, void *cbdata)
orte_process_name_t proc_name;
int rc=ORTE_SUCCESS;
orte_grpcomm_collective_t *modex = (orte_grpcomm_collective_t*)cbdata;
char *rml_uri;
OPAL_OUTPUT_VERBOSE((2, orte_grpcomm_base_framework.framework_output,
"%s STORING MODEX DATA",
@ -411,6 +424,16 @@ void orte_grpcomm_base_store_modex(opal_buffer_t *rbuf, void *cbdata)
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&proc_name)));
if (ORTE_PROC_IS_NON_MPI) {
cnt = 1;
if (ORTE_SUCCESS != (rc = opal_dss.unpack(rbuf, &rml_uri, &cnt, OPAL_STRING))) {
ORTE_ERROR_LOG(rc);
goto cleanup;
}
orte_rml.set_contact_info(rml_uri);
free(rml_uri);
}
/* update the modex database */
if (ORTE_SUCCESS != (rc = orte_grpcomm_base_update_modex_entries(&proc_name, rbuf))) {
ORTE_ERROR_LOG(rc);
@ -422,8 +445,6 @@ void orte_grpcomm_base_store_modex(opal_buffer_t *rbuf, void *cbdata)
}
cleanup:
/* flag the modex as complete */
modex->active = false;
/* cleanup */
opal_list_remove_item(&orte_grpcomm_base.active_colls, &modex->super);
/* execute user callback, if requested */
@ -433,6 +454,8 @@ void orte_grpcomm_base_store_modex(opal_buffer_t *rbuf, void *cbdata)
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
modex->cbfunc(NULL, modex->cbdata);
}
/* flag the modex as complete */
modex->active = false;
}

118
orte/mca/grpcomm/base/grpcomm_base_receive.c
Просмотреть файл

@ -60,68 +60,42 @@ static void coll_id_req(int status, orte_process_name_t* sender,
int orte_grpcomm_base_comm_start(void)
{
int rc;
OPAL_OUTPUT_VERBOSE((5, orte_grpcomm_base_framework.framework_output,
"%s grpcomm:base:receive start comm",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
if (!recv_issued) {
if (ORTE_PROC_IS_HNP || ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLLECTIVE,
ORTE_RML_PERSISTENT,
daemon_local_recv, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_XCAST,
ORTE_RML_PERSISTENT,
orte_grpcomm_base_xcast_recv, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DAEMON_COLL,
ORTE_RML_PERSISTENT,
daemon_coll_recv, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLLECTIVE,
ORTE_RML_PERSISTENT,
daemon_local_recv, NULL);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_XCAST,
ORTE_RML_PERSISTENT,
orte_grpcomm_base_xcast_recv, NULL);
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_DAEMON_COLL,
ORTE_RML_PERSISTENT,
daemon_coll_recv, NULL);
if (ORTE_PROC_IS_DAEMON) {
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_ROLLUP,
ORTE_RML_PERSISTENT,
orte_grpcomm_base_rollup_recv, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_ROLLUP,
ORTE_RML_PERSISTENT,
orte_grpcomm_base_rollup_recv, NULL);
}
if (ORTE_PROC_IS_HNP) {
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLL_ID_REQ,
ORTE_RML_PERSISTENT,
coll_id_req, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLL_ID_REQ,
ORTE_RML_PERSISTENT,
coll_id_req, NULL);
}
recv_issued = true;
} else if (ORTE_PROC_IS_APP) {
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLLECTIVE,
ORTE_RML_PERSISTENT,
app_recv, NULL))) {
ORTE_ERROR_LOG(rc);
recv_issued = false;
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_COLLECTIVE,
ORTE_RML_PERSISTENT,
app_recv, NULL);
recv_issued = true;
}
}
@ -167,7 +141,7 @@ static void coll_id_req(int status, orte_process_name_t* sender,
OBJ_RELEASE(relay);
return;
}
if (0 > (rc = orte_rml.send_buffer_nb(sender, relay, ORTE_RML_TAG_COLL_ID, 0,
if (0 > (rc = orte_rml.send_buffer_nb(sender, relay, ORTE_RML_TAG_COLL_ID,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(relay);
@ -186,6 +160,7 @@ static void app_recv(int status, orte_process_name_t* sender,
int n, rc;
orte_grpcomm_coll_id_t id;
orte_namelist_t *nm;
bool added;
/* get the collective id */
n = 1;
@ -220,10 +195,8 @@ static void app_recv(int status, orte_process_name_t* sender,
coll->next_cb(buffer, coll->next_cbdata);
break;
}
/* flag the collective as complete */
coll->active = false;
/* cleanup */
opal_list_remove_item(&orte_grpcomm_base.active_colls, item);
opal_list_remove_item(&orte_grpcomm_base.active_colls, &coll->super);
/* callback the specified function */
if (NULL != coll->cbfunc) {
OPAL_OUTPUT_VERBOSE((5, orte_grpcomm_base_framework.framework_output,
@ -232,6 +205,11 @@ static void app_recv(int status, orte_process_name_t* sender,
coll->cbfunc(buffer, coll->cbdata);
}
/* flag the collective as complete - must do this after we remove the
* item and do the callback because someone may be waiting inside
* a different event base
*/
coll->active = false;
/* do NOT release the collective - it is the responsibility
* of whomever passed it down to us
*/
@ -249,6 +227,7 @@ static void app_recv(int status, orte_process_name_t* sender,
* the collective on our list - see if we do
*/
coll = NULL;
added = false;
for (item = opal_list_get_first(&orte_grpcomm_base.active_colls);
item != opal_list_get_end(&orte_grpcomm_base.active_colls);
item = opal_list_get_next(item)) {
@ -269,7 +248,12 @@ static void app_recv(int status, orte_process_name_t* sender,
coll = OBJ_NEW(orte_grpcomm_collective_t);
coll->id = id;
opal_list_append(&orte_grpcomm_base.active_colls, &coll->super);
}
/* mark that we added it - since we can't possibly know
* the participants, we need to mark this collective so
* we don't try to test for completeness
*/
added = true;
}
/* append the sender to the list of targets so
* we know we already have their contribution
*/
@ -284,11 +268,12 @@ static void app_recv(int status, orte_process_name_t* sender,
*/
opal_dss.copy_payload(&coll->local_bucket, buffer);
/* if the length of the participant list equals the
* length of the target list, then the collective is
* complete
/* if we already know the participants, and the length of the
* participant list equals the length of the target list, then
* the collective is complete
*/
if (opal_list_get_size(&coll->participants) == opal_list_get_size(&coll->targets)) {
if (!added &&
opal_list_get_size(&coll->participants) == opal_list_get_size(&coll->targets)) {
/* replace whatever is in the collective's buffer
* field with what we collected
*/
@ -301,10 +286,8 @@ static void app_recv(int status, orte_process_name_t* sender,
coll->next_cb(&coll->buffer, coll->next_cbdata);
return;
}
/* flag the collective as complete */
coll->active = false;
/* cleanup */
opal_list_remove_item(&orte_grpcomm_base.active_colls, item);
opal_list_remove_item(&orte_grpcomm_base.active_colls, &coll->super);
/* callback the specified function */
if (NULL != coll->cbfunc) {
OPAL_OUTPUT_VERBOSE((5, orte_grpcomm_base_framework.framework_output,
@ -313,6 +296,11 @@ static void app_recv(int status, orte_process_name_t* sender,
coll->cbfunc(&coll->buffer, coll->cbdata);
}
/* flag the collective as complete - must do this after we remove the
* item and do the callback because someone may be waiting inside
* a different event base
*/
coll->active = false;
/* do NOT release the collective - it is the responsibility
* of whomever passed it down to us
*/
@ -449,7 +437,7 @@ void orte_grpcomm_base_progress_collectives(void)
coll, ORTE_GRPCOMM_INTERNAL_STG_LOCAL);
/* send it to our global collective handler */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_NAME, relay,
ORTE_RML_TAG_DAEMON_COLL, 0,
ORTE_RML_TAG_DAEMON_COLL,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(relay);
@ -594,7 +582,7 @@ static void daemon_coll_recv(int status, orte_process_name_t* sender,
if (nm->name.vpid == sender->vpid) {
OBJ_RELEASE(relay);
} else {
if (0 > orte_rml.send_buffer_nb(&nm->name, relay, ORTE_RML_TAG_DAEMON_COLL, 0,
if (0 > orte_rml.send_buffer_nb(&nm->name, relay, ORTE_RML_TAG_DAEMON_COLL,
orte_rml_send_callback, NULL)) {
ORTE_ERROR_LOG(ORTE_ERR_COMM_FAILURE);
OBJ_RELEASE(relay);
@ -655,7 +643,7 @@ static void daemon_coll_recv(int status, orte_process_name_t* sender,
OBJ_RELEASE(relay);
} else {
/* send it to this proc */
if (0 > orte_rml.send_buffer_nb(&nm->name, relay, ORTE_RML_TAG_COLLECTIVE, 0,
if (0 > orte_rml.send_buffer_nb(&nm->name, relay, ORTE_RML_TAG_COLLECTIVE,
orte_rml_send_callback, NULL)) {
ORTE_ERROR_LOG(ORTE_ERR_COMM_FAILURE);
OBJ_RELEASE(relay);

4
orte/mca/grpcomm/base/grpcomm_base_rollup.c
Просмотреть файл

@ -112,13 +112,13 @@ void orte_grpcomm_base_rollup_recv(int status, orte_process_name_t* sender,
/* if my parent is the HNP, send it to the final destination */
if (ORTE_PROC_MY_PARENT->vpid == ORTE_PROC_MY_HNP->vpid) {
if (0 > (ret = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, relay,
ORTE_RML_TAG_ORTED_CALLBACK, 0,
ORTE_RML_TAG_ORTED_CALLBACK,
rml_send_callback, NULL))) {
ORTE_ERROR_LOG(ret);
}
} else {
if (0 > (ret = orte_rml.send_buffer_nb(ORTE_PROC_MY_PARENT, relay,
ORTE_RML_TAG_ROLLUP, 0,
ORTE_RML_TAG_ROLLUP,
rml_send_callback, NULL))) {
ORTE_ERROR_LOG(ret);
}

13
orte/mca/grpcomm/base/grpcomm_base_xcast.c
Просмотреть файл

@ -36,6 +36,7 @@
#include "orte/mca/odls/base/base.h"
#include "orte/mca/rml/rml.h"
#include "orte/mca/routed/routed.h"
#include "orte/mca/state/state.h"
#include "orte/util/name_fns.h"
#include "orte/util/nidmap.h"
#include "orte/runtime/orte_globals.h"
@ -61,8 +62,9 @@ void orte_grpcomm_base_xcast_recv(int status, orte_process_name_t* sender,
orte_proc_t *rec;
OPAL_OUTPUT_VERBOSE((1, orte_grpcomm_base_framework.framework_output,
"%s grpcomm:xcast:recv:send_relay",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME)));
"%s grpcomm:xcast:recv: with %d bytes",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
(int)buffer->bytes_used));
/* setup the relay message */
relay = OBJ_NEW(opal_buffer_t);
@ -72,7 +74,8 @@ void orte_grpcomm_base_xcast_recv(int status, orte_process_name_t* sender,
cnt=1;
if (ORTE_SUCCESS != (ret = opal_dss.unpack(buffer, &command, &cnt, ORTE_DAEMON_CMD))) {
ORTE_ERROR_LOG(ret);
goto relay;
ORTE_FORCED_TERMINATE(ret);
return;
}
/* if it is add_procs, then... */
@ -184,7 +187,7 @@ void orte_grpcomm_base_xcast_recv(int status, orte_process_name_t* sender,
OBJ_RELEASE(rly);
continue;
}
if (0 > (ret = orte_rml.send_buffer_nb(&nm->name, rly, ORTE_RML_TAG_XCAST, 0,
if (0 > (ret = orte_rml.send_buffer_nb(&nm->name, rly, ORTE_RML_TAG_XCAST,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(ret);
OBJ_RELEASE(rly);
@ -198,7 +201,7 @@ void orte_grpcomm_base_xcast_recv(int status, orte_process_name_t* sender,
/* now send it to myself for processing */
if (0 > (ret = orte_rml.send_buffer_nb(ORTE_PROC_MY_NAME, relay,
ORTE_RML_TAG_DAEMON, 0,
ORTE_RML_TAG_DAEMON,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(ret);
OBJ_RELEASE(relay);

2
orte/mca/grpcomm/pmi/grpcomm_pmi_module.c
Просмотреть файл

@ -173,7 +173,7 @@ static int modex(orte_grpcomm_collective_t *coll)
{
int *local_ranks, local_rank_count;
opal_hwloc_locality_t locality;
const char *cpuset, *rmluri;
const char *cpuset;
orte_process_name_t name;
orte_vpid_t v;
bool local;

4
orte/mca/iof/base/iof_base_output.c
Просмотреть файл

@ -300,7 +300,7 @@ void orte_iof_base_write_handler(int fd, short event, void *cbdata)
/* if the list is getting too large, abort */
if (orte_iof_base.output_limit < opal_list_get_size(&wev->outputs)) {
opal_output(0, "IO Forwarding is running too far behind - something is blocking us from writing");
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto ABORT;
}
/* leave the write event running so it will call us again
@ -321,7 +321,7 @@ void orte_iof_base_write_handler(int fd, short event, void *cbdata)
/* if the list is getting too large, abort */
if (orte_iof_base.output_limit < opal_list_get_size(&wev->outputs)) {
opal_output(0, "IO Forwarding is running too far behind - something is blocking us from writing");
ORTE_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
ORTE_FORCED_TERMINATE(ORTE_ERROR_DEFAULT_EXIT_CODE);
goto ABORT;
}
/* leave the write event running so it will call us again

14
orte/mca/iof/hnp/iof_hnp.c
Просмотреть файл

@ -97,15 +97,11 @@ static int init(void)
/* post non-blocking recv to catch forwarded IO from
* the orteds
*/
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_HNP,
ORTE_RML_PERSISTENT,
orte_iof_hnp_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_HNP,
ORTE_RML_PERSISTENT,
orte_iof_hnp_recv,
NULL);
if (ORTE_SUCCESS != (rc = orte_rml.add_exception_handler(orte_iof_hnp_exception_handler))) {
ORTE_ERROR_LOG(rc);

15
orte/mca/iof/hnp/iof_hnp_send.c
Просмотреть файл

@ -10,6 +10,8 @@
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2012 Los Alamos National Security, LLC
* All rights reserved
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -40,17 +42,6 @@
#include "iof_hnp.h"
/*
* Callback when non-blocking RML send completes.
*/
static void send_cb(int status, orte_process_name_t *peer,
opal_buffer_t *buf, orte_rml_tag_t tag,
void *cbdata)
{
/* nothing to do here - just release buffer and return */
OBJ_RELEASE(buf);
}
int orte_iof_hnp_send_data_to_endpoint(orte_process_name_t *host,
orte_process_name_t *target,
orte_iof_tag_t tag,
@ -112,7 +103,7 @@ int orte_iof_hnp_send_data_to_endpoint(orte_process_name_t *host,
* a tool that requested IOF
*/
if (0 > (rc = orte_rml.send_buffer_nb(host, buf, ORTE_RML_TAG_IOF_PROXY,
0, send_cb, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}

18
orte/mca/iof/mr_hnp/iof_mrhnp.c
Просмотреть файл

@ -79,20 +79,14 @@ orte_iof_base_module_t orte_iof_mrhnp_module = {
/* Initialize the module */
static int init(void)
{
int rc;
/* post non-blocking recv to catch forwarded IO from
* the orteds
*/
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_HNP,
ORTE_RML_PERSISTENT,
orte_iof_mrhnp_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_HNP,
ORTE_RML_PERSISTENT,
orte_iof_mrhnp_recv,
NULL);
OBJ_CONSTRUCT(&mca_iof_mr_hnp_component.sinks, opal_list_t);
OBJ_CONSTRUCT(&mca_iof_mr_hnp_component.procs, opal_list_t);
@ -462,7 +456,7 @@ static void send_data(orte_process_name_t *name, orte_iof_tag_t tag,
}
if (0 > (rc = orte_rml.send_buffer_nb(name, buf, ORTE_RML_TAG_IOF_PROXY,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
}

2
orte/mca/iof/mr_hnp/iof_mrhnp_read.c
Просмотреть файл

@ -365,7 +365,7 @@ static void send_data(orte_process_name_t *name, orte_iof_tag_t tag,
}
if (0 > (rc = orte_rml.send_buffer_nb(name, buf, ORTE_RML_TAG_IOF_PROXY,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
}

21
orte/mca/iof/mr_orted/iof_mrorted.c
Просмотреть файл

@ -83,19 +83,13 @@ orte_iof_base_module_t orte_iof_mrorted_module = {
static int init(void)
{
int rc;
/* post a non-blocking RML receive to get messages
from the HNP IOF component */
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_mrorted_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_mrorted_recv,
NULL);
/* setup the local global variables */
OBJ_CONSTRUCT(&mca_iof_mr_orted_component.sinks, opal_list_t);
@ -349,7 +343,6 @@ static void mrorted_complete(const orte_job_t *jdata)
static int finalize(void)
{
int rc;
opal_list_item_t *item;
while ((item = opal_list_remove_first(&mca_iof_mr_orted_component.sinks)) != NULL) {
@ -361,8 +354,8 @@ static int finalize(void)
}
OBJ_DESTRUCT(&mca_iof_mr_orted_component.procs);
/* Cancel the RML receive */
rc = orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
return rc;
orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
return ORTE_SUCCESS;
}
/*

4
orte/mca/iof/mr_orted/iof_mrorted_read.c
Просмотреть файл

@ -187,7 +187,7 @@ void orte_iof_mrorted_read_handler(int fd, short event, void *cbdata)
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME), numbytes));
orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf, ORTE_RML_TAG_IOF_HNP,
0, orte_rml_send_callback, NULL);
orte_rml_send_callback, NULL);
}
/* re-add the event */
@ -265,7 +265,7 @@ static void send_data(orte_process_name_t *name, orte_iof_tag_t tag,
}
if (0 > (rc = orte_rml.send_buffer_nb(name, buf, ORTE_RML_TAG_IOF_PROXY,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buf);
}

2
orte/mca/iof/mr_orted/iof_mrorted_receive.c
Просмотреть файл

@ -68,7 +68,7 @@ void orte_iof_mrorted_send_xonxoff(orte_process_name_t *name, orte_iof_tag_t tag
/* send the buffer to the HNP */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf, ORTE_RML_TAG_IOF_HNP,
0, send_cb, NULL))) {
send_cb, NULL))) {
ORTE_ERROR_LOG(rc);
}
}

21
orte/mca/iof/orted/iof_orted.c
Просмотреть файл

@ -91,19 +91,13 @@ orte_iof_base_module_t orte_iof_orted_module = {
static int init(void)
{
int rc;
/* post a non-blocking RML receive to get messages
from the HNP IOF component */
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_orted_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_orted_recv,
NULL);
/* setup the local global variables */
OBJ_CONSTRUCT(&mca_iof_orted_component.sinks, opal_list_t);
@ -308,7 +302,6 @@ static int orted_close(const orte_process_name_t* peer,
static int finalize(void)
{
int rc;
opal_list_item_t *item;
while ((item = opal_list_remove_first(&mca_iof_orted_component.sinks)) != NULL) {
@ -320,8 +313,8 @@ static int finalize(void)
}
OBJ_DESTRUCT(&mca_iof_orted_component.procs);
/* Cancel the RML receive */
rc = orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
return rc;
orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
return ORTE_SUCCESS;
}
/*

2
orte/mca/iof/orted/iof_orted_read.c
Просмотреть файл

@ -160,7 +160,7 @@ void orte_iof_orted_read_handler(int fd, short event, void *cbdata)
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME), numbytes));
orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf, ORTE_RML_TAG_IOF_HNP,
0, send_cb, NULL);
send_cb, NULL);
RESTART:
/* re-add the event */

2
orte/mca/iof/orted/iof_orted_receive.c
Просмотреть файл

@ -72,7 +72,7 @@ void orte_iof_orted_send_xonxoff(orte_iof_tag_t tag)
/* send the buffer to the HNP */
if (0 > (rc = orte_rml.send_buffer_nb(ORTE_PROC_MY_HNP, buf, ORTE_RML_TAG_IOF_HNP,
0, send_cb, NULL))) {
send_cb, NULL))) {
ORTE_ERROR_LOG(rc);
}
}

27
orte/mca/iof/tool/iof_tool.c
Просмотреть файл

@ -69,20 +69,14 @@ orte_iof_base_module_t orte_iof_tool_module = {
static int init(void)
{
int rc;
{
/* post a non-blocking RML receive to get messages
from the HNP IOF component */
if (ORTE_SUCCESS != (rc = orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_tool_recv,
NULL))) {
ORTE_ERROR_LOG(rc);
return rc;
}
orte_rml.recv_buffer_nb(ORTE_NAME_WILDCARD,
ORTE_RML_TAG_IOF_PROXY,
ORTE_RML_PERSISTENT,
orte_iof_tool_recv,
NULL);
mca_iof_tool_component.closed = false;
@ -168,7 +162,7 @@ static int tool_pull(const orte_process_name_t* src_name,
/* send the buffer to the correct HNP */
ORTE_HNP_NAME_FROM_JOB(&hnp, src_name->jobid);
orte_rml.send_buffer_nb(&hnp, buf, ORTE_RML_TAG_IOF_HNP,
0, send_cb, NULL);
send_cb, NULL);
return ORTE_SUCCESS;
}
@ -217,14 +211,13 @@ static int tool_close(const orte_process_name_t* src_name,
/* send the buffer to the correct HNP */
ORTE_HNP_NAME_FROM_JOB(&hnp, src_name->jobid);
orte_rml.send_buffer_nb(&hnp, buf, ORTE_RML_TAG_IOF_HNP,
0, send_cb, NULL);
send_cb, NULL);
return ORTE_SUCCESS;
}
static int finalize(void)
{
int rc;
opal_list_item_t* item;
orte_iof_write_output_t *output;
orte_iof_write_event_t *wev;
@ -271,9 +264,9 @@ static int finalize(void)
}
/* Cancel the RML receive */
rc = orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
orte_rml.recv_cancel(ORTE_NAME_WILDCARD, ORTE_RML_TAG_IOF_PROXY);
return rc;
return ORTE_SUCCESS;
}
/*

11
orte/mca/odls/base/odls_base_default_fns.c
Просмотреть файл

@ -1628,7 +1628,7 @@ int orte_odls_base_default_deliver_message(orte_jobid_t job, opal_buffer_t *buff
/* if so, send the message */
relay = OBJ_NEW(opal_buffer_t);
opal_dss.copy_payload(relay, buffer);
rc = orte_rml.send_buffer_nb(&child->name, relay, tag, 0, orte_rml_send_callback, NULL);
rc = orte_rml.send_buffer_nb(&child->name, relay, tag, orte_rml_send_callback, NULL);
if (rc < 0 && rc != ORTE_ERR_ADDRESSEE_UNKNOWN) {
/* ignore if the addressee is unknown as a race condition could
* have allowed the child to exit before we send it a barrier
@ -1810,7 +1810,7 @@ int orte_odls_base_default_require_sync(orte_process_name_t *proc,
ORTE_NAME_PRINT(proc), (long)buffer->bytes_used));
if (0 > (rc = orte_rml.send_buffer_nb(proc, buffer, ORTE_RML_TAG_SYNC,
0, orte_rml_send_callback, NULL))) {
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
goto CLEANUP;
@ -1862,7 +1862,7 @@ void orte_odls_base_default_report_abort(orte_process_name_t *proc)
/* send ack */
buffer = OBJ_NEW(opal_buffer_t);
if (0 > (rc = orte_rml.send_buffer_nb(proc, buffer,
ORTE_RML_TAG_ABORT, 0,
ORTE_RML_TAG_ABORT,
orte_rml_send_callback, NULL))) {
ORTE_ERROR_LOG(rc);
OBJ_RELEASE(buffer);
@ -1970,11 +1970,12 @@ void odls_base_default_wait_local_proc(pid_t pid, int status, void* cbdata)
/* check to see if a sync was required and if it was received */
if (proc->registered) {
if (proc->deregistered || orte_allowed_exit_without_sync) {
if (proc->deregistered || orte_allowed_exit_without_sync || 0 != proc->exit_code) {
/* if we did recv a finalize sync, or one is not required,
* then declare it normally terminated
* unless it returned with a non-zero status indicating the code
* felt it was non-normal
* felt it was non-normal - in this latter case, we do not
* require that the proc deregister before terminating
*/
if (0 != proc->exit_code) {
state = ORTE_PROC_STATE_TERM_NON_ZERO;

8
orte/mca/oob/base/Makefile.am
Просмотреть файл

@ -9,7 +9,8 @@
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2013 Los Alamos National Security, LLC. All rights reserved.
# Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
# reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
@ -18,8 +19,9 @@
#
headers += \
base/base.h
base/base.h
libmca_oob_la_SOURCES += \
base/oob_base_stubs.c \
base/oob_base_frame.c \
base/oob_base_init.c
base/oob_base_select.c

133
orte/mca/oob/base/base.h
Просмотреть файл

@ -9,7 +9,8 @@
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2013 Los Alamos National Security, LLC. All rights reserved.
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -36,28 +37,138 @@
#include <net/uio.h>
#endif
#include "orte/mca/oob/oob.h"
#include "opal/class/opal_bitmap.h"
#include "opal/class/opal_hash_table.h"
#include "opal/class/opal_list.h"
#include "opal/mca/mca.h"
#include "opal/mca/event/event.h"
#include "orte/mca/oob/oob.h"
BEGIN_C_DECLS
/*
* MCA framework
* Convenience Typedef
*/
typedef struct {
char *include;
char *exclude;
opal_list_t components;
opal_list_t actives;
int max_uri_length;
opal_hash_table_t peers;
bool use_module_threads;
} orte_oob_base_t;
ORTE_DECLSPEC extern orte_oob_base_t orte_oob_base;
typedef struct {
opal_object_t super;
mca_oob_base_component_t *component;
opal_bitmap_t addressable;
} orte_oob_base_peer_t;
OBJ_CLASS_DECLARATION(orte_oob_base_peer_t);
/* MCA framework */
ORTE_DECLSPEC extern mca_base_framework_t orte_oob_base_framework;
ORTE_DECLSPEC int orte_oob_base_select(void);
/*
* Global functions for MCA overall collective open and close
/* Access the OOB internal functions via set of event-based macros
* for inserting messages and other commands into the
* OOB event base. This ensures that all OOB operations occur
* asynchronously in a thread-safe environment.
* Note that this doesn't mean that messages will be *sent*
* in order as that depends on the specific transport being
* used, when that module's event base indicates the transport
* is available, etc.
*/
typedef struct {
opal_object_t super;
opal_event_t ev;
orte_rml_send_t *msg;
} orte_oob_send_t;
OBJ_CLASS_DECLARATION(orte_oob_send_t);
/* All OOB sends are based on iovec's and are async as the RML
* acts as the initial interface to prepare all communications.
* The send_nb function will enter the message into the OOB
* base, which will then check to see if a transport for the
* intended target has already been assigned. If so, the message
* is immediately placed into that module's event base for
* transmission. If not, the function will loop across all available
* components until one identifies that it has a module capable
* of reaching the target.
*/
ORTE_DECLSPEC int mca_oob_base_init(void);
typedef void (*mca_oob_send_callback_fn_t)(int status,
struct iovec *iov,
int count, void *cbdata);
ORTE_DECLSPEC void orte_oob_base_send_nb(int fd, short args, void *cbdata);
#define ORTE_OOB_SEND(m) \
do { \
orte_oob_send_t *cd; \
cd = OBJ_NEW(orte_oob_send_t); \
cd->msg = (m); \
opal_event_set(orte_event_base, &cd->ev, -1, \
OPAL_EV_WRITE, \
orte_oob_base_send_nb, cd); \
opal_event_set_priority(&cd->ev, ORTE_MSG_PRI); \
opal_event_active(&cd->ev, OPAL_EV_WRITE, 1); \
}while(0);
/*
* Global struct holding the selected module's function pointers
/* Our contact info is actually subject to change as transports
* can fail at any time. So a request to obtain our URI requires
* that we get a snapshot in time. Since the request always comes
* thru the rml, and we share that event base, we can just cycle
* across the components to collect the info.
*
* During initial wireup, we can only transfer contact info on the daemon
* command line. This limits what we can send to a string representation of
* the actual contact info, which gets sent in a uri-like form. Not every
* oob module can support this transaction, so this function will loop
* across all oob components/modules, letting each add to the uri string if
* it supports bootstrap operations. An error will be returned in the cbfunc
* if NO component can successfully provide a contact.
*
* Note: since there is a limit to what an OS will allow on a cmd line, we
* impose a limit on the length of the resulting uri via an MCA param. The
* default value of -1 implies unlimited - however, users with large numbers
* of interfaces on their nodes may wish to restrict the size.
*
* Since all components define their address info at component start,
* it is unchanged and does not require acess via event
*/
extern char* mca_oob_base_include;
extern char* mca_oob_base_exclude;
#define ORTE_OOB_GET_URI(u) orte_oob_base_get_addr(u)
ORTE_DECLSPEC void orte_oob_base_get_addr(char **uri);
/**
* Extract initial contact information from a string uri
*
* During initial wireup, we can only transfer contact info on the daemon
* command line. This limits what we can send to a string representation of
* the actual contact info, which gets sent in a uri-like form. Not every
* oob module can support this transaction, so this function will loop
* across all oob components/modules, letting each look at the uri and extract
* info from it if it can.
*/
typedef struct {
opal_object_t super;
opal_event_t ev;
char *uri;
} mca_oob_uri_req_t;
OBJ_CLASS_DECLARATION(mca_oob_uri_req_t);
#define ORTE_OOB_SET_URI(u) \
do { \
mca_oob_uri_req_t *rq; \
rq = OBJ_NEW(mca_oob_uri_req_t); \
rq->uri = strdup((u)); \
opal_event_set(orte_event_base, &(rq)->ev, -1, \
OPAL_EV_WRITE, \
orte_oob_base_set_addr, (rq)); \
opal_event_set_priority(&(rq)->ev, ORTE_MSG_PRI); \
opal_event_active(&(rq)->ev, OPAL_EV_WRITE, 1); \
}while(0);
ORTE_DECLSPEC void orte_oob_base_set_addr(int fd, short args, void *cbdata);
END_C_DECLS
#endif

60
orte/mca/oob/base/oob_base_frame.c
Просмотреть файл

@ -22,11 +22,12 @@
#include "orte_config.h"
#include "orte/constants.h"
#include "opal/class/opal_bitmap.h"
#include "opal/mca/mca.h"
#include "opal/util/output.h"
#include "opal/mca/base/base.h"
#include "orte/mca/rml/base/base.h"
#include "orte/mca/oob/base/base.h"
/*
@ -40,14 +41,36 @@
/*
* Global variables
*/
mca_oob_t mca_oob;
orte_oob_base_t orte_oob_base;
static int orte_oob_base_register(mca_base_register_flag_t flags)
{
(void)mca_base_var_register("orte", "oob", "base", "enable_module_progress_threads",
"Whether to independently progress OOB messages for each interface",
MCA_BASE_VAR_TYPE_BOOL, NULL, 0, 0,
OPAL_INFO_LVL_9,
MCA_BASE_VAR_SCOPE_READONLY,
&orte_oob_base.use_module_threads);
return ORTE_SUCCESS;
}
static int orte_oob_base_close(void)
{
if (NULL != mca_oob.oob_fini) {
mca_oob.oob_fini();
mca_oob_base_component_t *component;
mca_base_component_list_item_t *cli;
/* shutdown all active transports */
OPAL_LIST_FOREACH(cli, &orte_oob_base.actives, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t*)cli->cli_component;
if (NULL != component->shutdown) {
component->shutdown();
}
}
/* destruct our internal lists */
OBJ_DESTRUCT(&orte_oob_base.actives);
return mca_base_framework_components_close(&orte_oob_base_framework, NULL);
}
@ -57,9 +80,36 @@ static int orte_oob_base_close(void)
*/
static int orte_oob_base_open(mca_base_open_flag_t flags)
{
/* setup globals */
orte_oob_base.max_uri_length = -1;
OBJ_CONSTRUCT(&orte_oob_base.peers, opal_hash_table_t);
opal_hash_table_init(&orte_oob_base.peers, 128);
OBJ_CONSTRUCT(&orte_oob_base.actives, opal_list_t);
/* Open up all available components */
return mca_base_framework_components_open(&orte_oob_base_framework, flags);
}
MCA_BASE_FRAMEWORK_DECLARE(orte, oob, NULL, NULL, orte_oob_base_open, orte_oob_base_close,
MCA_BASE_FRAMEWORK_DECLARE(orte, oob, "Out-of-Band Messaging Subsystem",
orte_oob_base_register, orte_oob_base_open, orte_oob_base_close,
mca_oob_base_static_components, 0);
OBJ_CLASS_INSTANCE(mca_oob_base_component_t,
opal_list_item_t,
NULL, NULL);
static void pr_cons(orte_oob_base_peer_t *ptr)
{
ptr->component = NULL;
OBJ_CONSTRUCT(&ptr->addressable, opal_bitmap_t);
opal_bitmap_init(&ptr->addressable, 8);
}
static void pr_des(orte_oob_base_peer_t *ptr)
{
OBJ_DESTRUCT(&ptr->addressable);
}
OBJ_CLASS_INSTANCE(orte_oob_base_peer_t,
opal_object_t,
pr_cons, pr_des);

102
orte/mca/oob/base/oob_base_init.c
Просмотреть файл

@ -1,102 +0,0 @@
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2006 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "orte_config.h"
#include "orte/constants.h"
#include <stdio.h>
#include <string.h>
#include "opal/mca/mca.h"
#include "opal/util/output.h"
#include "opal/mca/base/base.h"
#include "orte/runtime/orte_globals.h"
#include "orte/mca/oob/oob.h"
#include "orte/mca/oob/base/base.h"
OBJ_CLASS_INSTANCE(
mca_oob_t,
opal_list_item_t,
NULL,
NULL
);
/**
* Function for selecting one module from all those that are
* available.
*
* Call the init function on all available modules.
*/
int mca_oob_base_init(void)
{
opal_list_item_t *item;
mca_base_component_list_item_t *cli;
mca_oob_base_component_t *component, *s_component=NULL;
mca_oob_t *module;
mca_oob_t *s_module = NULL;
int s_priority = -1;
/* Traverse the list of available modules; call their init functions. */
for (item = opal_list_get_first(&orte_oob_base_framework.framework_components);
item != opal_list_get_end(&orte_oob_base_framework.framework_components);
item = opal_list_get_next(item)) {
cli = (mca_base_component_list_item_t *) item;
component = (mca_oob_base_component_t *) cli->cli_component;
if (NULL == component->oob_init) {
opal_output_verbose(10, orte_oob_base_framework.framework_output,
"mca_oob_base_init: no init function; ignoring component");
} else {
int priority = -1;
module = component->oob_init(&priority);
if (NULL != module) {
if (priority > s_priority) {
s_module = module;
s_priority = priority;
s_component = component;
} else {
if (NULL != module->oob_fini) {
module->oob_fini();
}
}
}
}
}
/* set the global variable to point to the first initialize module */
if (s_module == NULL) {
opal_output_verbose(10, orte_oob_base_framework.framework_output,
"mca_oob_base_init: no OOB modules available\n");
/* the oob modules will have printed out an error msg - so be silent here */
return ORTE_ERR_SILENT;
}
mca_oob = *s_module;
orte_selected_oob_component = strdup(s_component->oob_base.mca_component_name);
opal_output_verbose(10, orte_oob_base_framework.framework_output,
"mca_oob_base_init: %s module selected\n",
s_component->oob_base.mca_component_name);
return ORTE_SUCCESS;
}

133
orte/mca/oob/base/oob_base_select.c Обычный файл
Просмотреть файл

@ -0,0 +1,133 @@
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2006 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "orte_config.h"
#include "orte/constants.h"
#include <stdio.h>
#include <string.h>
#include "opal/mca/mca.h"
#include "opal/util/output.h"
#include "opal/mca/base/base.h"
#include "orte/runtime/orte_globals.h"
#include "orte/mca/oob/oob.h"
#include "orte/mca/oob/base/base.h"
/**
* Function for selecting all runnable modules from those that are
* available.
*
* Call the init function on all available modules.
*/
int orte_oob_base_select(void)
{
mca_base_component_list_item_t *cli, *cmp, *c2;
mca_oob_base_component_t *component, *c3;
bool added;
int i;
/* Query all available components and ask if their transport is available */
OPAL_LIST_FOREACH(cli, &orte_oob_base_framework.framework_components, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t *) cli->cli_component;
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: checking available component %s",
component->oob_base.mca_component_name);
/* If there's no query function, skip it */
if (NULL == component->available) {
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Skipping component [%s]. It does not implement a query function",
component->oob_base.mca_component_name );
continue;
}
/* Query the component */
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Querying component [%s]",
component->oob_base.mca_component_name);
/* If the component is not available, then skip it as
* it has no available interfaces
*/
if (!component->available()) {
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Skipping component [%s] - no available interfaces",
component->oob_base.mca_component_name );
continue;
}
/* if it fails to startup, then skip it */
if (ORTE_SUCCESS != component->startup()) {
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Skipping component [%s] - failed to startup",
component->oob_base.mca_component_name );
continue;
}
/* record it, but maintain priority order */
added = false;
OPAL_LIST_FOREACH(cmp, &orte_oob_base.actives, mca_base_component_list_item_t) {
c3 = (mca_oob_base_component_t *) cmp->cli_component;
if (c3->priority < component->priority) {
continue;
}
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Inserting component");
c2 = OBJ_NEW(mca_base_component_list_item_t);
c2->cli_component = (mca_base_component_t*)component;
opal_list_insert_pos(&orte_oob_base.actives,
&c2->super, &cmp->super);
added = true;
break;
}
if (!added) {
/* add to end */
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Adding component to end");
c2 = OBJ_NEW(mca_base_component_list_item_t);
c2->cli_component = (mca_base_component_t*)component;
opal_list_append(&orte_oob_base.actives, &c2->super);
}
}
if (0 == opal_list_get_size(&orte_oob_base.actives)) {
/* no support available means we really cannot run */
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Init failed to return any available transports");
return ORTE_ERR_NOT_AVAILABLE;
}
/* provide them an index so we can track their usability in a bitmap */
i=0;
OPAL_LIST_FOREACH(cmp, &orte_oob_base.actives, mca_base_component_list_item_t) {
c3 = (mca_oob_base_component_t *) cmp->cli_component;
c3->idx = i++;
}
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"mca:oob:select: Found %d active transports",
(int)opal_list_get_size(&orte_oob_base.actives));
return ORTE_SUCCESS;
}

388
orte/mca/oob/base/oob_base_stubs.c Обычный файл
Просмотреть файл

@ -0,0 +1,388 @@
/*
* Copyright (c) 2012-2013 Los Alamos National Security, LLC. All rights
* reserved.
* Copyright (c) 2013 Intel, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "orte_config.h"
#include "orte/constants.h"
#include "opal/util/output.h"
#include "opal/mca/db/db.h"
#include "orte/mca/errmgr/errmgr.h"
#include "orte/mca/state/state.h"
#include "orte/mca/rml/rml.h"
#include "orte/mca/oob/base/base.h"
OBJ_CLASS_INSTANCE(orte_oob_send_t,
opal_object_t,
NULL, NULL);
static void process_uri(char *uri);
void orte_oob_base_send_nb(int fd, short args, void *cbdata)
{
orte_oob_send_t *cd = (orte_oob_send_t*)cbdata;
orte_rml_send_t *msg = cd->msg;
mca_base_component_list_item_t *cli;
orte_oob_base_peer_t *pr;
int rc;
uint64_t ui64;
bool msg_sent;
mca_oob_base_component_t *component;
bool reachable;
char *rmluri;
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"%s oob:base:send to target %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&msg->peer));
/* check if we have this peer in our hash table */
memcpy(&ui64, (char*)&msg->peer, sizeof(uint64_t));
if (OPAL_SUCCESS != opal_hash_table_get_value_uint64(&orte_oob_base.peers,
ui64, (void**)&pr) ||
NULL == pr) {
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"%s oob:base:send unknown peer %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&msg->peer));
/* for direct launched procs, the URI might be in the database,
* so check there next - if it is, the peer object will be added
* to our hash table
*/
if (OPAL_SUCCESS == opal_db.fetch_pointer((opal_identifier_t*)&msg->peer, ORTE_DB_RMLURI, (void **)&rmluri, OPAL_STRING)) {
process_uri(rmluri);
free(rmluri);
if (OPAL_SUCCESS != opal_hash_table_get_value_uint64(&orte_oob_base.peers,
ui64, (void**)&pr) ||
NULL == pr) {
/* that is just plain wrong */
ORTE_ERROR_LOG(ORTE_ERR_ADDRESSEE_UNKNOWN);
msg->status = ORTE_ERR_ADDRESSEE_UNKNOWN;
ORTE_RML_SEND_COMPLETE(msg);
OBJ_RELEASE(cd);
return;
}
} else {
/* even though we don't know about this peer yet, we still might
* be able to get to it via routing, so ask each component if
* it can reach it
*/
reachable = false;
OPAL_LIST_FOREACH(cli, &orte_oob_base.actives, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t*)cli->cli_component;
if (NULL != component->is_reachable) {
if (component->is_reachable(&msg->peer)) {
/* there is a way to reach this peer - record it
* so we don't waste this time again
*/
pr = OBJ_NEW(orte_oob_base_peer_t);
if (OPAL_SUCCESS != (rc = opal_hash_table_set_value_uint64(&orte_oob_base.peers, ui64, (void*)pr))) {
ORTE_ERROR_LOG(rc);
msg->status = ORTE_ERR_ADDRESSEE_UNKNOWN;
ORTE_RML_SEND_COMPLETE(msg);
OBJ_RELEASE(cd);
return;
}
/* mark that this component can reach the peer */
opal_bitmap_set_bit(&pr->addressable, component->idx);
/* flag that at least one component can reach this peer */
reachable = true;
}
}
}
/* if nobody could reach it, then that's an error */
if (!reachable) {
msg->status = ORTE_ERR_ADDRESSEE_UNKNOWN;
ORTE_RML_SEND_COMPLETE(msg);
OBJ_RELEASE(cd);
return;
}
}
}
/* if we already have a connection to this peer, use it */
if (NULL != pr->component) {
/* post this msg for send by this transport - the component
* runs on our event base, so we can just call their function
*/
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"%s oob:base:send known transport for peer %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&msg->peer));
if (ORTE_SUCCESS == (rc = pr->component->send_nb(msg))) {
OBJ_RELEASE(cd);
return;
}
}
/* if we haven't identified a transport to this peer,
* loop across all available components in priority order until
* one replies that it has a module that can reach this peer.
* Let it try to make the connection
*/
msg_sent = false;
OPAL_LIST_FOREACH(cli, &orte_oob_base.actives, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t*)cli->cli_component;
/* is this peer addressable by this component? */
if (!opal_bitmap_is_set_bit(&pr->addressable, component->idx)) {
continue;
}
/* it is addressable, so attempt to send via that transport */
if (ORTE_SUCCESS == (rc = component->send_nb(msg))) {
/* the msg status will be set upon send completion/failure */
msg_sent = true;
/* point to this transport for any future messages */
pr->component = component;
break;
} else if (ORTE_ERR_TAKE_NEXT_OPTION != rc) {
/* components return "next option" if they can't connect
* to this peer. anything else is a true error.
*/
ORTE_ERROR_LOG(rc);
msg->status = rc;
ORTE_RML_SEND_COMPLETE(msg);
OBJ_RELEASE(cd);
return;
}
}
/* if no component can reach this peer, that's an error - post
* it back to the RML for handling
*/
if (!msg_sent) {
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"%s oob:base:send no path to target %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
ORTE_NAME_PRINT(&msg->peer));
msg->status = ORTE_ERR_NO_PATH_TO_TARGET;
ORTE_RML_SEND_COMPLETE(msg);
}
OBJ_RELEASE(cd);
}
/**
* Obtain a uri for initial connection purposes
*
* During initial wireup, we can only transfer contact info on the daemon
* command line. This limits what we can send to a string representation of
* the actual contact info, which gets sent in a uri-like form. Not every
* oob module can support this transaction, so this function will loop
* across all oob components/modules, letting each add to the uri string if
* it supports bootstrap operations. An error will be returned in the cbfunc
* if NO component can successfully provide a contact.
*
* Note: since there is a limit to what an OS will allow on a cmd line, we
* impose a limit on the length of the resulting uri via an MCA param. The
* default value of -1 implies unlimited - however, users with large numbers
* of interfaces on their nodes may wish to restrict the size.
*/
void orte_oob_base_get_addr(char **uri)
{
char *turi, *final=NULL, *tmp;
size_t len = 0;
int rc=ORTE_SUCCESS;
bool one_added = false;
mca_base_component_list_item_t *cli;
mca_oob_base_component_t *component;
/* start with our process name */
if (ORTE_SUCCESS != (rc = orte_util_convert_process_name_to_string(&final, ORTE_PROC_MY_NAME))) {
ORTE_ERROR_LOG(rc);
goto unblock;
}
len = strlen(final);
/* loop across all available modules to get their input
* up to the max length
*/
OPAL_LIST_FOREACH(cli, &orte_oob_base.actives, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t*)cli->cli_component;
/* ask the component for its input, obtained when it
* opened its modules
*/
if (NULL == component->get_addr) {
/* doesn't support this ability */
continue;
}
/* the components operate within our event base, so we
* can directly call their get_uri function to get the
* pointer to the uri - this is not a copy, so
* do NOT free it!
*/
turi = component->get_addr();
/* check overall length for limits */
if (0 < orte_oob_base.max_uri_length &&
orte_oob_base.max_uri_length < (int)(len + strlen(turi))) {
/* cannot accept the payload */
continue;
}
/* add new value to final one */
asprintf(&tmp, "%s;%s", final, turi);
free(turi);
free(final);
final = tmp;
len = strlen(final);
/* flag that at least one contributed */
one_added = true;
}
if (!one_added) {
/* nobody could contribute */
if (NULL != final) {
free(final);
final = NULL;
}
}
unblock:
*uri = final;
}
/**
* This function will loop
* across all oob components, letting each look at the uri and extract
* info from it if it can. An error is to be returned if NO component
* can successfully extract a contact.
*/
static void req_cons(mca_oob_uri_req_t *ptr)
{
ptr->uri = NULL;
}
static void req_des(mca_oob_uri_req_t *ptr)
{
if (NULL != ptr->uri) {
free(ptr->uri);
}
}
OBJ_CLASS_INSTANCE(mca_oob_uri_req_t,
opal_object_t,
req_cons, req_des);
void orte_oob_base_set_addr(int fd, short args, void *cbdata)
{
mca_oob_uri_req_t *req = (mca_oob_uri_req_t*)cbdata;
char *uri = req->uri;
opal_output_verbose(5, orte_oob_base_framework.framework_output,
"%s: set_addr to uri %s",
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME),
(NULL == uri) ? "NULL" : uri);
/* if the request doesn't contain a URI, then we
* have an error
*/
if (NULL == uri) {
opal_output(0, "%s: NULL URI", ORTE_NAME_PRINT(ORTE_PROC_MY_NAME));
ORTE_FORCED_TERMINATE(1);
OBJ_RELEASE(req);
return;
}
process_uri(uri);
OBJ_RELEASE(req);
}
static void process_uri(char *uri)
{
orte_process_name_t peer;
char *cptr;
mca_base_component_list_item_t *cli;
mca_oob_base_component_t *component;
char **uris=NULL;
int rc;
uint64_t ui64;
orte_oob_base_peer_t *pr;
/* find the first semi-colon in the string */
cptr = strchr(uri, ';');
if (NULL == cptr) {
/* got a problem - there must be at least two fields,
* the first containing the process name of our peer
* and all others containing the OOB contact info
*/
ORTE_ERROR_LOG(ORTE_ERR_BAD_PARAM);
return;
}
*cptr = '\0';
cptr++;
/* the first field is the process name, so convert it */
orte_util_convert_string_to_process_name(&peer, uri);
/* if the peer is us, no need to go further as we already
* know our own contact info
*/
if (peer.jobid == ORTE_PROC_MY_NAME->jobid &&
peer.vpid == ORTE_PROC_MY_NAME->vpid) {
return;
}
/* split the rest of the uri into component parts */
uris = opal_argv_split(cptr, ';');
/* get the peer object for this process */
memcpy(&ui64, (char*)&peer, sizeof(uint64_t));
if (OPAL_SUCCESS != opal_hash_table_get_value_uint64(&orte_oob_base.peers,
ui64, (void**)&pr) ||
NULL == pr) {
pr = OBJ_NEW(orte_oob_base_peer_t);
if (OPAL_SUCCESS != (rc = opal_hash_table_set_value_uint64(&orte_oob_base.peers, ui64, (void*)pr))) {
ORTE_ERROR_LOG(rc);
opal_argv_free(uris);
return;
}
}
/* loop across all available components and let them extract
* whatever piece(s) of the uri they find relevant - they
* are all operating on our event base, so we can just
* directly call their functions
*/
rc = ORTE_ERR_UNREACH;
OPAL_LIST_FOREACH(cli, &orte_oob_base.actives, mca_base_component_list_item_t) {
component = (mca_oob_base_component_t*)cli->cli_component;
if (NULL != component->set_addr) {
if (ORTE_SUCCESS == component->set_addr(&peer, uris)) {
/* this component found reachable addresses
* in the uris
*/
opal_bitmap_set_bit(&pr->addressable, component->idx);
}
}
}
opal_argv_free(uris);
}
#if 0
int mca_oob_base_ft_event(int state)
{
int rc;
mca_oob_module_t *mod;
/* loop across all available modules in priority order
* and call each one's ft_event handler
*/
OPAL_LIST_FOREACH(mod, &mca_oob_base_modules, mca_oob_module_t) {
if (NULL != mod->ft_event) {
if (ORTE_SUCCESS != (rc = mod->ft_event(state))) {
ORTE_ERROR_LOG(rc);
}
}
}
return ORTE_SUCCESS;
}
#endif

180
orte/mca/oob/oob.h
Просмотреть файл

@ -9,6 +9,8 @@
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2012 Los Alamos National Security, LLC. All rights
* reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
@ -29,153 +31,44 @@
#ifdef HAVE_UNISTD_H
#include <unistd.h>
#endif
#ifdef HAVE_SYS_UIO_H
#include <sys/uio.h>
#endif
#ifdef HAVE_NET_UIO_H
#include <net/uio.h>
#endif
#include "opal/class/opal_list.h"
#include "opal/class/opal_pointer_array.h"
#include "opal/mca/mca.h"
#include "orte/mca/rml/rml.h"
#include "opal/mca/crs/crs.h"
#include "opal/mca/crs/base/base.h"
#include "orte/mca/rml/base/base.h"
BEGIN_C_DECLS
typedef bool (*mca_oob_base_component_avail_fn_t)(void);
typedef int (*mca_oob_base_component_startup_fn_t)(void);
typedef void (*mca_oob_base_component_shutdown_fn_t)(void);
typedef int (*mca_oob_base_component_send_fn_t)(orte_rml_send_t *msg);
typedef char* (*mca_oob_base_component_get_addr_fn_t)(void);
typedef int (*mca_oob_base_component_set_addr_fn_t)(orte_process_name_t *peer,
char **uris);
typedef bool (*mca_oob_base_component_is_reachable_fn_t)(orte_process_name_t *peer);
typedef void (*mca_oob_ping_callback_fn_t)(int status, void *cbdata);
struct mca_oob_1_0_0_t;
typedef struct mca_oob_1_0_0_t mca_oob_1_0_0_t;
typedef struct mca_oob_1_0_0_t mca_oob_t;
typedef mca_oob_t* (*mca_oob_base_component_init_fn_t)(int *priority);
struct mca_oob_base_component_2_0_0_t {
mca_base_component_t oob_base;
mca_base_component_data_t oob_data;
mca_oob_base_component_init_fn_t oob_init;
};
typedef struct mca_oob_base_component_2_0_0_t mca_oob_base_component_2_0_0_t;
typedef mca_oob_base_component_2_0_0_t mca_oob_base_component_t;
typedef char* (*mca_oob_base_module_get_addr_fn_t)(void);
typedef int (*mca_oob_base_module_set_addr_fn_t)(const orte_process_name_t* peer,
const char* uri);
/**
* Implementation of mca_oob_ping().
*
* @param peer (IN) Opaque name of peer process.
* @param tv (IN) Timeout to wait in connection response.
* @return OMPI error code (<0) or ORTE_SUCCESS
*/
typedef int (*mca_oob_base_module_ping_fn_t)(const orte_process_name_t*,
const char* uri,
const struct timeval* tv);
/**
* Send an oob message
*
* Send an oob message. All oob sends are non-blocking, and cbfunc
* will be called when the message has been sent. When cbfunc is
* called, message has been injected into the network but no guarantee
* is made about whether the target has received the message.
*
* @param[in] target Destination process name
* @param[in] origin Origin process for the message, for the purposes
* of message matching. This can be different from
* the process calling send().
* @param[in] msg Array of iovecs describing user buffers and lengths.
* @param[in] count Number of elements in iovec array.
* @param[in] tag User defined tag for matching send/recv.
* @param[in] flags Currently unused.
* @param[in] cbfunc Callback function on send completion.
* @param[in] cbdata User data that is passed to callback function.
*
* @return OMPI error code (<0) on error number of bytes actually sent.
*/
typedef int (*mca_oob_base_module_send_nb_fn_t)(
orte_process_name_t* target,
orte_process_name_t* origin,
struct iovec* msg,
int count,
int tag,
int flags,
orte_rml_callback_fn_t cbfunc,
void* cbdata);
/**
* Implementation of mca_oob_recv_nb().
*
* @param peer (IN) Opaque name of peer process or ORTE_NAME_WILDCARD for wildcard receive.
* @param msg (IN) Array of iovecs describing user buffers and lengths.
* @param count (IN) Number of elements in iovec array.
* @param tag (IN) User defined tag for matching send/recv.
* @param flags (IN) May be MCA_OOB_PEEK to return up to size bytes of msg w/out removing it from the queue,
* @param cbfunc (IN) Callback function on recv completion.
* @param cbdata (IN) User data that is passed to callback function.
* @return OMPI error code (<0) on error or number of bytes actually received.
*/
typedef int (*mca_oob_base_module_recv_nb_fn_t)(
orte_process_name_t* peer,
struct iovec* msg,
int count,
int tag,
int flags,
orte_rml_callback_fn_t cbfunc,
void* cbdata);
/**
* Implementation of mca_oob_recv_cancel().
*
* @param peer (IN) Opaque name of peer process or ORTE_NAME_WILDCARD for wildcard receive.
* @param tag (IN) User defined tag for matching send/recv.
* @return OMPI error code (<0) on error or number of bytes actually received.
*/
typedef int (*mca_oob_base_module_recv_cancel_fn_t)(orte_process_name_t* peer, int tag);
/**
* Hook function called by mca_oob_base_register to allow
* the oob component a chance to register contact information
*/
typedef int (*mca_oob_base_module_init_fn_t)(void);
/**
* Cleanup during finalize.
*/
typedef int (*mca_oob_base_module_fini_fn_t)(void);
/* ft event */
typedef int (*mca_oob_base_module_ft_event_fn_t)( int state );
/**
* OOB Module
*/
struct mca_oob_1_0_0_t {
mca_oob_base_module_init_fn_t oob_init;
mca_oob_base_module_fini_fn_t oob_fini;
mca_oob_base_module_get_addr_fn_t oob_get_addr;
mca_oob_base_module_set_addr_fn_t oob_set_addr;
mca_oob_base_module_ping_fn_t oob_ping;
mca_oob_base_module_send_nb_fn_t oob_send_nb;
mca_oob_base_module_recv_nb_fn_t oob_recv_nb;
mca_oob_base_module_recv_cancel_fn_t oob_recv_cancel;
mca_oob_base_module_ft_event_fn_t oob_ft_event;
orte_rml_exception_callback_t oob_exception_callback;
};
typedef struct {
mca_base_component_t oob_base;
mca_base_component_data_t oob_data;
int idx;
int priority;
mca_oob_base_component_avail_fn_t available;
mca_oob_base_component_startup_fn_t startup;
mca_oob_base_component_shutdown_fn_t shutdown;
mca_oob_base_component_send_fn_t send_nb;
mca_oob_base_component_get_addr_fn_t get_addr;
mca_oob_base_component_set_addr_fn_t set_addr;
mca_oob_base_component_is_reachable_fn_t is_reachable;
} mca_oob_base_component_t;
/**
* Macro for use in components that are of type oob
@ -184,13 +77,6 @@ struct mca_oob_1_0_0_t {
MCA_BASE_VERSION_2_0_0, \
"oob", 2, 0, 0
/*
* BWB - FIX ME - This is the first module on the list. This is here
* temporarily to make things work
*/
ORTE_DECLSPEC extern mca_oob_t mca_oob;
END_C_DECLS
#endif

31
orte/mca/oob/tcp/Makefile.am
Просмотреть файл

@ -10,6 +10,8 @@
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2010 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2012-2013 Los Alamos National Security, LLC.
# All rights reserved
# $COPYRIGHT$
#
# Additional copyrights may follow
@ -17,21 +19,22 @@
# $HEADER$
#
dist_pkgdata_DATA = help-oob-tcp.txt
sources = \
oob_tcp.c \
oob_tcp.h \
oob_tcp_hdr.h \
oob_tcp_addr.h \
oob_tcp_addr.c \
oob_tcp_msg.c \
oob_tcp_msg.h \
oob_tcp_peer.c \
oob_tcp_ping.c \
oob_tcp_peer.h \
oob_tcp_recv.c \
oob_tcp_send.c
oob_tcp_component.h \
oob_tcp.h \
oob_tcp_listener.h \
oob_tcp_common.h \
oob_tcp_component.c \
oob_tcp_connection.h \
oob_tcp_sendrecv.h \
oob_tcp_hdr.h \
oob_tcp_peer.h \
oob_tcp_ping.h \
oob_tcp.c \
oob_tcp_listener.c \
oob_tcp_common.c \
oob_tcp_connection.c \
oob_tcp_sendrecv.c
# Make the output library in this directory, and name it either
# mca_<type>_<name>.la (for DSO builds) or libmca_<type>_<name>.la

2524
orte/mca/oob/tcp/oob_tcp.c
Просмотреть файл

Разница между файлами не показана из-за своего большого размера Загрузить разницу

312
orte/mca/oob/tcp/oob_tcp.h
Просмотреть файл

@ -18,268 +18,98 @@
*
* $HEADER$
*/
/** @file:
*
* Defines the functions for the tcp module.
*/
#ifndef _MCA_OOB_TCP_H_
#define _MCA_OOB_TCP_H_
#include "orte_config.h"
#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
#endif
#ifdef HAVE_SYS_SOCKET_H
#include <sys/socket.h>
#endif
#include "orte/types.h"
#include "opal/mca/base/base.h"
#include "opal/class/opal_free_list.h"
#include "opal/class/opal_hash_table.h"
#include "opal/threads/mutex.h"
#include "opal/threads/condition.h"
#include "opal/threads/threads.h"
#include "opal/mca/timer/base/base.h"
#include "opal/mca/event/event.h"
#include "orte/mca/oob/oob.h"
#include "orte/mca/oob/base/base.h"
#include "orte/mca/oob/tcp/oob_tcp_peer.h"
#include "orte/mca/oob/tcp/oob_tcp_msg.h"
BEGIN_C_DECLS
/* define some debug levels */
#define OOB_TCP_DEBUG_FAIL 2
#define OOB_TCP_DEBUG_CONNECT 7
#define OOB_TCP_DEBUG_CONNECT_FAIL 1 /* debug connection establishment failures */
#define OOB_TCP_DEBUG_CONNECT 2 /* other connection information */
#define OOB_TCP_DEBUG_INFO 3 /* information about startup, connection establish, etc. */
#define OOB_TCP_DEBUG_ALL 4 /* everything else */
/* forward declare a couple of structures */
struct mca_oob_tcp_module_t;
struct mca_oob_tcp_msg_error_t;
extern mca_oob_t mca_oob_tcp;
/*
* standard component functions
*/
mca_oob_t* mca_oob_tcp_component_init(int* priority);
/**
* Hook function to allow the selected oob components
* to register their contact info with the registry
*/
int mca_oob_tcp_init(void);
/**
* Cleanup resources during shutdown.
*/
int mca_oob_tcp_fini(void);
/**
* Compare two process names for equality.
*
* @param n1 Process name 1.
* @param n2 Process name 2.
* @return (-1 for n1<n2 0 for equality, 1 for n1>n2)
*
* Note that the definition of < or > is somewhat arbitrary -
* just needs to be consistently applied to maintain an ordering
* when process names are used as indices.
*/
int mca_oob_tcp_process_name_compare(const orte_process_name_t* n1, const orte_process_name_t* n2);
/**
* Obtain contact information for this host (e.g. <ipaddress>:<port>)
*/
char* mca_oob_tcp_get_addr(void);
/**
* Setup cached addresses for the peers.
*/
int mca_oob_tcp_set_addr(const orte_process_name_t*, const char*);
int mca_oob_tcp_get_new_name(orte_process_name_t* name);
/**
* A routine to ping a given process name to determine if it is reachable.
*
* @param name The peer name.
* @param tv The length of time to wait on a connection/response.
*
* Note that this routine blocks up to the specified timeout waiting for a
* connection / response from the specified peer. If the peer is unavailable
* an error status is returned.
*/
int mca_oob_tcp_ping(const orte_process_name_t*, const char* uri, const struct timeval* tv);
/*
* Non-blocking versions of send/recv.
*/
int mca_oob_tcp_send_nb(orte_process_name_t* target,
orte_process_name_t* origin,
struct iovec* msg,
int count,
int tag,
int flags,
orte_rml_callback_fn_t cbfunc,
void* cbdata);
/**
* Non-blocking version of mca_oob_recv().
*
* @param peer (IN) Opaque name of peer process or ORTE_NAME_WILDCARD for wildcard receive.
* @param msg (IN) Array of iovecs describing user buffers and lengths.
* @param count (IN) Number of elements in iovec array.
* @param tag (IN) User defined tag for matching send/recv.
* @param flags (IN) May be MCA_OOB_PEEK to return up to size bytes of msg w/out removing it from the queue,
* @param cbfunc (IN) Callback function on recv completion.
* @param cbdata (IN) User data that is passed to callback function.
* @return OMPI error code (<0) on error or number of bytes actually received.
*/
int mca_oob_tcp_recv_nb(
orte_process_name_t* peer,
struct iovec* msg,
int count,
int tag,
int flags,
orte_rml_callback_fn_t cbfunc,
void* cbdata);
/**
* Cancel non-blocking receive.
*
* @param peer (IN) Opaque name of peer process or ORTE_NAME_WILDCARD for wildcard receive.
* @param tag (IN) User defined tag for matching send/recv.
* @return OMPI error code (<0) on error or number of bytes actually received.
*/
int mca_oob_tcp_recv_cancel(
orte_process_name_t* peer,
int tag);
/**
* Attempt to map a peer name to its corresponding address.
*/
int mca_oob_tcp_resolve(mca_oob_tcp_peer_t*);
/**
* Parse a URI string into an IP address and port number.
*/
int mca_oob_tcp_parse_uri(
const char* uri,
struct sockaddr* inaddr
);
/**
* Setup socket options
*/
void mca_oob_tcp_set_socket_options(int sd);
int mca_oob_tcp_ft_event(int state);
typedef enum { OOB_TCP_EVENT, OOB_TCP_LISTEN_THREAD } mca_oob_tcp_listen_type_t;
/**
* OOB TCP Component
*/
struct mca_oob_tcp_component_t {
mca_oob_base_component_2_0_0_t super; /**< base OOB component */
char* tcp_include; /**< list of ip interfaces to include */
char* tcp_exclude; /**< list of ip interfaces to exclude */
opal_list_t tcp_peer_list; /**< list of peers sorted in mru order */
opal_hash_table_t tcp_peers; /**< peers sorted by name */
opal_hash_table_t tcp_peer_names; /**< cache of peer contact info sorted by name */
opal_free_list_t tcp_peer_free; /**< free list of peers */
int tcp_peer_limit; /**< max size of tcp peer cache */
int tcp_peer_retries; /**< max number of retries before declaring peer gone */
int tcp_sndbuf; /**< socket send buffer size */
int tcp_rcvbuf; /**< socket recv buffer size */
opal_free_list_t tcp_msgs; /**< free list of messages */
opal_event_t *tcp_recv_event; /**< event structure for IPv4 recvs */
int tcp_listen_sd; /**< listen socket for incoming IPv4 connection requests */
unsigned short tcp_listen_port; /**< IPv4 listen port */
char** tcp4_static_ports; /**< Static ports - IPV4 */
char** tcp4_dyn_ports; /**< Dynamic ports - IPV4 */
int disable_family; /**< disable AF: 0-nothing, 4-IPv4, 6-IPv6 */
#if OPAL_WANT_IPV6
opal_event_t *tcp6_recv_event; /**< event structure for IPv6 recvs */
int tcp6_listen_sd; /**< listen socket for incoming IPv6 connection requests */
unsigned short tcp6_listen_port; /**< IPv6 listen port */
char** tcp6_static_ports; /**< Static ports - IPV6 */
char** tcp6_dyn_ports; /**< Dynamic ports - IPV6 */
#endif /* OPAL_WANT_IPV6 */
opal_mutex_t tcp_lock; /**< lock for accessing module state */
opal_list_t tcp_events; /**< list of pending events (accepts) */
opal_list_t tcp_msg_post; /**< list of recieves user has posted */
opal_list_t tcp_msg_recv; /**< list of recieved messages */
opal_list_t tcp_msg_completed; /**< list of completed messages */
opal_mutex_t tcp_match_lock; /**< lock held while searching/posting messages */
opal_condition_t tcp_match_cond; /**< condition variable used in finalize */
int tcp_match_count; /**< number of matched recvs in progress */
int tcp_debug; /**< debug level */
bool tcp_shutdown;
mca_oob_tcp_listen_type_t tcp_listen_type;
opal_list_t tcp_available_devices;
opal_thread_t tcp_listen_thread; /** handle to the listening thread */
opal_list_t tcp_pending_connections; /**< List of accepted connections waiting for processing */
opal_list_t tcp_connections_return; /**< List of connection fragments being returned to accept thread */
opal_mutex_t tcp_connections_lock; /**< Lock protecting pending_connections and connections_return */
int tcp_connections_pipe[2];
opal_event_t *tcp_listen_thread_event;
int tcp_copy_max_size; /**< Max size of the copy list before copying must commence */
int tcp_listen_thread_num_sockets; /**< Number of sockets in tcp_listen_thread_sds */
int tcp_listen_thread_sds[2]; /**< Room for IPv4 and IPv6. Might need to make more dynamic. */
struct timeval tcp_listen_thread_tv; /**< Timeout when using listen thread */
int connect_sleep;
};
/**
* Convenience Typedef
*/
typedef struct mca_oob_tcp_component_t mca_oob_tcp_component_t;
ORTE_MODULE_DECLSPEC extern mca_oob_tcp_component_t mca_oob_tcp_component;
extern int mca_oob_tcp_output_handle;
#define CLOSE_THE_SOCKET(socket) \
do { \
shutdown(socket, 2); \
close(socket); \
} while(0)
struct mca_oob_tcp_pending_connection_t {
opal_free_list_item_t super;
int fd;
struct sockaddr_storage addr;
};
typedef struct mca_oob_tcp_pending_connection_t mca_oob_tcp_pending_connection_t;
OBJ_CLASS_DECLARATION(mca_oob_tcp_pending_connection_t);
struct mca_oob_tcp_device_t {
/* define a struct for tracking NIC addresses */
typedef struct {
opal_list_item_t super;
int if_index;
bool if_local;
struct sockaddr_storage if_addr;
};
typedef struct mca_oob_tcp_device_t mca_oob_tcp_device_t;
OBJ_CLASS_DECLARATION(mca_oob_tcp_device_t);
uint16_t af_family;
struct sockaddr addr;
} mca_oob_tcp_nicaddr_t;
OBJ_CLASS_DECLARATION(mca_oob_tcp_nicaddr_t);
/* Module definition */
typedef void (*mca_oob_tcp_module_init_fn_t)(struct mca_oob_tcp_module_t *mod);
typedef void (*mca_oob_tcp_module_fini_fn_t)(struct mca_oob_tcp_module_t *mod);
typedef void (*mca_oob_tcp_module_accept_connection_fn_t)(struct mca_oob_tcp_module_t *md,
const int accepted_fd,
const struct sockaddr *addr);
typedef void (*mca_oob_tcp_module_set_peer_fn_t)(struct mca_oob_tcp_module_t *mod,
const orte_process_name_t* name,
const uint16_t af_family,
const char *net, const char *ports);
typedef void (*mca_oob_tcp_module_ping_fn_t)(struct mca_oob_tcp_module_t *mod,
const orte_process_name_t *proc);
typedef void (*mca_oob_tcp_module_send_nb_fn_t)(struct mca_oob_tcp_module_t *mod,
orte_rml_send_t *msg);
typedef void (*mca_oob_tcp_module_resend_nb_fn_t)(struct mca_oob_tcp_msg_error_t *mop);
typedef void (*mca_oob_tcp_module_ft_event_fn_t)(struct mca_oob_tcp_module_t *mod, int state);
typedef struct {
mca_oob_tcp_module_init_fn_t init;
mca_oob_tcp_module_fini_fn_t finalize;
mca_oob_tcp_module_accept_connection_fn_t accept_connection;
mca_oob_tcp_module_set_peer_fn_t set_peer;
mca_oob_tcp_module_ping_fn_t ping;
mca_oob_tcp_module_send_nb_fn_t send_nb;
mca_oob_tcp_module_resend_nb_fn_t resend;
mca_oob_tcp_module_ft_event_fn_t ft_event;
} mca_oob_tcp_module_api_t;
typedef struct {
mca_oob_tcp_module_api_t api;
int idx; // index in the module array
opal_event_base_t *ev_base; /* event base for the module progress thread */
bool ev_active;
opal_thread_t progress_thread;
int af_family; // interface family - v4 or v6
char* if_name; /* string name of the interface */
int if_kidx; /* interface kernel index */
opal_list_t addresses; /* list of addresses served by this NIC */
opal_hash_table_t peers; // connection addresses for peers
} mca_oob_tcp_module_t;
ORTE_MODULE_DECLSPEC extern mca_oob_tcp_module_t mca_oob_tcp_module;
/**
* the state of the connection
*/
typedef enum {
MCA_OOB_TCP_UNCONNECTED,
MCA_OOB_TCP_CLOSED,
MCA_OOB_TCP_RESOLVE,
MCA_OOB_TCP_CONNECTING,
MCA_OOB_TCP_CONNECT_ACK,
MCA_OOB_TCP_CONNECTED,
MCA_OOB_TCP_FAILED
} mca_oob_tcp_state_t;
/* module-level shared functions */
ORTE_MODULE_DECLSPEC void mca_oob_tcp_send_handler(int fd, short args, void *cbdata);
ORTE_MODULE_DECLSPEC void mca_oob_tcp_recv_handler(int fd, short args, void *cbdata);
END_C_DECLS

180
orte/mca/oob/tcp/oob_tcp_addr.c
Просмотреть файл

@ -1,180 +0,0 @@
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2005 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "orte_config.h"
#include "orte/constants.h"
#include "orte/types.h"
#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
#endif
#ifdef HAVE_SYS_SOCKET_H
#include <sys/socket.h>
#endif
#ifdef HAVE_NETINET_IN_H
#include <netinet/in.h>
#endif
#ifdef HAVE_ARPA_INET_H
#include <arpa/inet.h>
#endif
#include <string.h>
#include "opal/util/if.h"
#include "opal/util/net.h"
#include "oob_tcp.h"
#include "oob_tcp_addr.h"
static void mca_oob_tcp_addr_construct(mca_oob_tcp_addr_t* addr)
{
memset(&addr->addr_name, 0, sizeof(addr->addr_name));
addr->addr_count = 0;
addr->addr_alloc = 0;
addr->addr_next = 0;
addr->addr_inet = NULL;
addr->addr_matched = MCA_OOB_TCP_ADDR_UNCLASSIFIED;
}
static void mca_oob_tcp_addr_destruct(mca_oob_tcp_addr_t* addr)
{
if(addr->addr_inet != NULL)
free(addr->addr_inet);
}
OBJ_CLASS_INSTANCE(
mca_oob_tcp_addr_t,
opal_object_t,
mca_oob_tcp_addr_construct,
mca_oob_tcp_addr_destruct);
int mca_oob_tcp_addr_get_next(mca_oob_tcp_addr_t* addr, struct sockaddr* retval)
{
static uint32_t i_have = MCA_OOB_TCP_ADDR_UNCLASSIFIED; /* my own capabilities */
if((NULL == addr) || (0 == addr->addr_count)) {
return ORTE_ERROR;
}
if(MCA_OOB_TCP_ADDR_UNCLASSIFIED == addr->addr_matched) {
orte_std_cntr_t i=0;
for(i=addr->addr_next; i<addr->addr_count; i++) {
opal_list_item_t *item;
for (item = opal_list_get_first(&mca_oob_tcp_component.tcp_available_devices) ;
item != opal_list_get_end(&mca_oob_tcp_component.tcp_available_devices) ;
item = opal_list_get_next(item)) {
mca_oob_tcp_device_t *dev = (mca_oob_tcp_device_t*) item;
uint32_t inmask;
opal_ifindextomask(dev->if_index, &inmask, sizeof(inmask));
/* Decide which address to try first; note that we're
called multiple times and each time we need to
present a different address
Precedence rules:
- IPv4public has the highest priority
- when IPv4private + IPv6, use IPv6 (this should
be changed when there is something like a CellID)
*/
if (true == opal_net_addr_isipv4public ((struct sockaddr*) &dev->if_addr)) {
i_have |= MCA_OOB_TCP_ADDR_IPV4public;
}
if (true == opal_net_addr_isipv4public ((struct sockaddr*)&addr->addr_inet[i])) {
addr->addr_matched |= MCA_OOB_TCP_ADDR_IPV4public;
}
if ((MCA_OOB_TCP_ADDR_IPV4public ==
(i_have & MCA_OOB_TCP_ADDR_IPV4public)) &&
(MCA_OOB_TCP_ADDR_IPV4public ==
(addr->addr_matched & MCA_OOB_TCP_ADDR_IPV4public))) {
addr->addr_next = i;
goto done;
}
if (AF_INET6 == dev->if_addr.ss_family) {
i_have |= MCA_OOB_TCP_ADDR_IPV6;
}
if (AF_INET6 ==
((struct sockaddr_in6*)&addr->addr_inet[i])->sin6_family) {
addr->addr_matched |= MCA_OOB_TCP_ADDR_IPV6;
addr->addr_next = i;
goto done;
}
/* if match on network prefix - start here */
/* Bug, FIXME: This code is dangerous, it will prefer
local addresses even if they point to wrong hosts
(the multicluster problem).
We need more magic to select the best address
adi@2006-09-30
*/
if(opal_net_samenetwork((struct sockaddr*) &dev->if_addr,
(struct sockaddr*)&addr->addr_inet[i],
inmask)) {
addr->addr_matched |= MCA_OOB_TCP_ADDR_MATCHED;
addr->addr_next = i;
goto done;
}
}
}
done:
; /* NOP */
}
if (addr->addr_inet[addr->addr_next].ss_family == AF_INET) {
memcpy(retval, &addr->addr_inet[addr->addr_next],
sizeof(struct sockaddr_in));
} else {
memcpy(retval, &addr->addr_inet[addr->addr_next],
sizeof(struct sockaddr_in6));
}
if(++addr->addr_next >= addr->addr_count)
addr->addr_next = 0;
return ORTE_SUCCESS;
}
int
mca_oob_tcp_addr_insert(mca_oob_tcp_addr_t* addr, const struct sockaddr* inaddr)
{
if(addr->addr_alloc == 0) {
addr->addr_alloc = 2;
addr->addr_inet = (struct sockaddr_storage*) malloc(addr->addr_alloc * sizeof(struct sockaddr_storage));
} else if(addr->addr_count == addr->addr_alloc) {
addr->addr_alloc <<= 1;
addr->addr_inet = (struct sockaddr_storage*) realloc(addr->addr_inet, addr->addr_alloc * sizeof(struct sockaddr_storage));
}
if(NULL == addr->addr_inet) return ORTE_ERR_OUT_OF_RESOURCE;
if (inaddr->sa_family == AF_INET) {
memcpy(addr->addr_inet+addr->addr_count, inaddr, sizeof(struct sockaddr_in));
} else {
memcpy(addr->addr_inet+addr->addr_count, inaddr, sizeof(struct sockaddr_in6));
}
addr->addr_count++;
return ORTE_SUCCESS;
}

84
orte/mca/oob/tcp/oob_tcp_addr.h
Просмотреть файл

@ -1,84 +0,0 @@
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2006 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2010 Cisco Systems, Inc. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
/** @file:
*
* Contains header used by tcp oob.
*/
#ifndef _MCA_OOB_TCP_ADDR_H_
#define _MCA_OOB_TCP_ADDR_H_
#include "orte_config.h"
#include "orte/types.h"
#ifdef HAVE_SYS_TYPES_H
#include <sys/types.h>
#endif
#ifdef HAVE_SYS_SOCKET_H
#include <sys/socket.h>
#endif
#ifdef HAVE_NETINET_IN_H
#include <netinet/in.h>
#endif
#include "opal/class/opal_object.h"
BEGIN_C_DECLS
#define MCA_OOB_TCP_ADDR_UNCLASSIFIED 0 /* we don't know anything */
#define MCA_OOB_TCP_ADDR_MATCHED 1 /* peer has IP on the same LAN */
#define MCA_OOB_TCP_ADDR_IPV6 2 /* peer has an IPv6 address */
#define MCA_OOB_TCP_ADDR_IPV4public 4 /* peer has public IPv4 address */
#define MCA_OOB_TCP_ADDR_TYPE_AFINET 0x01
#define MCA_OOB_TCP_ADDR_TYPE_AFINET6 0x02
/**
* Address info published to registry
*/
struct mca_oob_tcp_addr_t {
opal_object_t super;
orte_process_name_t addr_name;
orte_std_cntr_t addr_count;
orte_std_cntr_t addr_next;
orte_std_cntr_t addr_alloc;
orte_std_cntr_t addr_matched;/* status of already tried address classes */
struct sockaddr_storage *addr_inet; /* yes, we want storage here, so the indexes work out... */
};
typedef struct mca_oob_tcp_addr_t mca_oob_tcp_addr_t;
OBJ_CLASS_DECLARATION(mca_oob_tcp_addr_t);
/**
*
*/
int mca_oob_tcp_addr_insert(mca_oob_tcp_addr_t*, const struct sockaddr*);
/**
*
*/
int mca_oob_tcp_addr_get_next(mca_oob_tcp_addr_t*, struct sockaddr*);
END_C_DECLS
#endif

Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше