1
1
openmpi/orte/util/Makefile.am
Ralph Castain ea35e47228 Fat SMPs (i.e., systems with nodes containing large numbers of cpus) were failing to start due to connection failures of the opal/pmix support. Root cause was that (a) we were setting the client socket to non-blocking before calling connect, and (b) the server was using the event library to harvest the accepts, and also did the handshake while in that event. So the server would backup beyond the connection backlog limit, and we would fail.
Changing the client to leave its socket as blocking during the connect doesn't solve the problem by itself - you also have to introduce a sleep delay once the backlog is hit to avoid simply machine-gunning your way thru retries. This gets somewhat difficult to adjust as you don't want to unnecessarily prolong startup time.

We've solved this before by adding a listening thread that simply reaps accepts and shoves them into the event library for subsequent processing. This would resolve the problem, but meant yet another daemon-level thread. So I centralized the listening thread support and let multiple elements register listeners on it. Thus, each daemon now has a single listening thread that reaps accepts from multiple sources - for now, the orte/pmix server and the oob/usock support are using it. I'll add in the oob/tcp component later.

This still didn't fully resolve the SMP problem, especially on coprocessor cards (e.g., KNC). Removing the shared memory dstore support helped further improve the behavior - it looks like there is some kind of memory paging issue there that needs further understanding. Given that the shared memory support was about to be lost when I bring over the PMIx integration (until it is restored in that library), it seemed like a reasonable thing to just remove it at this point.
2015-05-29 14:37:14 -07:00

84 строки
2.7 KiB
Makefile

#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2008 Sun Microsystems, Inc. All rights reserved.
# Copyright (c) 2014 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2014-2015 Intel, Inc. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# This makefile.am does not stand on its own - it is included from orte/Makefile.am
include $(top_srcdir)/Makefile.ompi-rules
dist_ortedata_DATA += util/hostfile/help-hostfile.txt \
util/dash_host/help-dash-host.txt \
util/help-regex.txt
nodist_man_MANS = util/hostfile/orte_hosts.7
# We are using $(am_dirstamp) instead of creating our own dirstamp since there
# is src code in util/hostfile directory is created. The automake process
# creates the $(am_dirstamp), we found the use of this in the generated Makefile
# in the util/Makefile
$(nodist_man_MANS): util/hostfile/$(am__dirstamp) $(top_builddir)/opal/include/opal_config.h
EXTRA_DIST += $(nodist_man_MANS:.7=.7in)
AM_LFLAGS = -Porte_util_hostfile_
LEX_OUTPUT_ROOT = lex.orte_util_hostfile_
headers += \
util/name_fns.h \
util/proc_info.h \
util/session_dir.h \
util/show_help.h \
util/error_strings.h \
util/context_fns.h \
util/parse_options.h \
util/pre_condition_transports.h \
util/hnp_contact.h \
util/hostfile/hostfile.h \
util/hostfile/hostfile_lex.h \
util/dash_host/dash_host.h \
util/comm/comm.h \
util/nidmap.h \
util/regex.h \
util/attr.h \
util/listener.h
lib@ORTE_LIB_PREFIX@open_rte_la_SOURCES += \
util/error_strings.c \
util/name_fns.c \
util/proc_info.c \
util/session_dir.c \
util/show_help.c \
util/context_fns.c \
util/parse_options.c \
util/pre_condition_transports.c \
util/hnp_contact.c \
util/hostfile/hostfile_lex.l \
util/hostfile/hostfile.c \
util/dash_host/dash_host.c \
util/comm/comm.c \
util/nidmap.c \
util/regex.c \
util/attr.c \
util/listener.c
# Remove the generated man pages
distclean-local:
rm -f $(nodist_man_MANS)