WHAT: Merge the PMIx branch into the devel repo, creating a new
OPAL “lmix” framework to abstract PMI support for all RTEs.
Replace the ORTE daemon-level collectives with a new PMIx
server and update the ORTE grpcomm framework to support
server-to-server collectives
WHY: We’ve had problems dealing with variations in PMI implementations,
and need to extend the existing PMI definitions to meet exascale
requirements.
WHEN: Mon, Aug 25
WHERE: https://github.com/rhc54/ompi-svn-mirror.git
Several community members have been working on a refactoring of the current PMI support within OMPI. Although the APIs are common, Slurm and Cray implement a different range of capabilities, and package them differently. For example, Cray provides an integrated PMI-1/2 library, while Slurm separates the two and requires the user to specify the one to be used at runtime. In addition, several bugs in the Slurm implementations have caused problems requiring extra coding.
All this has led to a slew of #if’s in the PMI code and bugs when the corner-case logic for one implementation accidentally traps the other. Extending this support to other implementations would have increased this complexity to an unacceptable level.
Accordingly, we have:
* created a new OPAL “pmix” framework to abstract the PMI support, with separate components for Cray, Slurm PMI-1, and Slurm PMI-2 implementations.
* Replaced the current ORTE grpcomm daemon-based collective operation with an integrated PMIx server, and updated the grpcomm APIs to provide more flexible, multi-algorithm support for collective operations. At this time, only the xcast and allgather operations are supported.
* Replaced the current global collective id with a signature based on the names of the participating procs. The allows an unlimited number of collectives to be executed by any group of processes, subject to the requirement that only one collective can be active at a time for a unique combination of procs. Note that a proc can be involved in any number of simultaneous collectives - it is the specific combination of procs that is subject to the constraint
* removed the prior OMPI/OPAL modex code
* added new macros for executing modex send/recv to simplify use of the new APIs. The send macros allow the caller to specify whether or not the BTL supports async modex operations - if so, then the non-blocking “fence” operation is used, if the active PMIx component supports it. Otherwise, the default is a full blocking modex exchange as we currently perform.
* retained the current flag that directs us to use a blocking fence operation, but only to retrieve data upon demand
This commit was SVN r32570.
Thanks to Ashley Pittman for pointing the modex should now be zero'ed
cmr=v1.8.2:ticket=trac:4871
This commit was SVN r32568.
The following Trac tickets were found above:
Ticket 4871 --> https://svn.open-mpi.org/trac/ompi/ticket/4871
Decrease the hash table size when an element is removed
cmr=v1.8.2:ticket=trac:4872
This commit was SVN r32566.
The following Trac tickets were found above:
Ticket 4872 --> https://svn.open-mpi.org/trac/ompi/ticket/4872
PGI compilers 2013 and older do not support the following syntax :
mca_btl_scif_modex_t modex = {.port_id = mca_btl_scif_module.port_id};
so split it on two lines
cmr=v1.8.2:reviewer=hjelmn
This commit was SVN r32555.
It turns out that we ''can'' get to the endpoint destructor with the
endpoint still on the "endpoints needing ACKs" list. So if it's on
the list, remove it first, and then DESTRUCT the opal_list_item_t.
This prevents an assert() fail in debug builds. We'd like to let this
soak over the weekend.
cmr=v1.8.2:reviewer=dgoodell
This commit was SVN r32546.
Rarely -- but it happens -- the connectivity client gets ECONNREFUSED
because the connectivity agent listen() backlog is too small. Rather
than put in a loop on the client side, take the simple way out for
now: increase the backlog size to an arbitrarily-large number.
Reviewed by Dave Goodell.
cmr=v1.8.2:reviewer=ompi-rm1.8
This commit was SVN r32543.
Instead of waiting to destroy the connectivity agent during component
shutdown, have the module shutdown send an "unlisten" command to the
cagent that will tell it to stop listening on a given interface.
This commit was SVN r32536.
1. After we receive N abnormally-short messages (meaning: corrupted),
print a show_help message about it. N defaults to 25. N can be set
to 0 disable the message via btl_usnic_max_short_packets.
1. If we receive a completion error for something other than a
receive, display a show_help message.
Reviewed by Dave Goodell.
CMR'ing to v1.8.3, but it will require a custom patch because of the
OMPI->OPAL BTL move.
cmr=v1.8.3
This commit was SVN r32522.
use HAVE_STRINGS_H to protect <strings.h> include
cmr=v1.8.2:reviewer=rhc:ticket=trac:4853
This commit was SVN r32510.
The following Trac tickets were found above:
Ticket 4853 --> https://svn.open-mpi.org/trac/ompi/ticket/4853
There were remaining references to size of MPI_COMM_WORLD,
etc. in ugni btl which prevented building of opal library
following btl move to opal.
Eventually another mechanism will need to be found for
providing hints to BTLs about how to setup internal
resources relevant to max. number possible endpoints, etc.
Conflicts:
opal/mca/btl/ugni/btl_ugni_component.c
This commit was SVN r32507.
Show an example of using the btl_usnic_connectivity_map option. Also,
mention that another reason for the "total connectivity failure" may
be due to asymmetric / unexpected routing.
Reviewed by Dave Goodell.
cmr=v1.8.2:reviewer=ompi-rm1.8
This commit was SVN r32465.
The contrib/check-help-strings.pl gets confused if the topic is an
inline logic check, so separate it into two calls to show_help.
This commit was SVN r32455.
The _strdup usage in opal/util/basename looks like it was a product of
Windows compatibility (see r11336), which we don't care about any
more. Further, opal/win32/win_compat.h, which we sitll maintain for
cygwin compatibility, #define's strdup to _strdup (which is what
Microsoft wants you to use).
So this old _strdup in opal/util/basename.c (and its corresponding
check in configure.ac) should just be removed.
This commit was SVN r32450.
The following SVN revision numbers were found above:
r11336 --> open-mpi/ompi@a28b025150
r32449 --> open-mpi/ompi@d5a3448b8b
_strdup is not part of any include file i could find on Solaris 10.
manually add the _strdup prototype if needed.
cmr=v1.8.2:reviewer=jsquyres
This commit was SVN r32449.
In core library portions of the configury (e.g., top-level
configure.ac itself), we were calling AC_CHECK_LIB and
OPAL_CHECK_FUNC_LIB to check for various libraries.
'''SIDENOTE:''' It turns out that modern Autoconf has AC_SEARCH_LIBS,
which does just about exactly what OPAL_CHECK_FUNC_LIB does. So this
commit effectively replaces OPAL_CHECK_FUNC_LIB with AC_SEARCH_LIBS.
However, we never bothered to add these found libraries to the wrapper
compiler list of libraries used for static linking (doh!). We've been
getting lucky for quite a while that components were adding the same
libraries to their wrapper compiler LIBS list.
This is problematic, however, if we don't build some of these
components. For example, Paul Hargrove noticed that if he configured
with --disable-shared --enable-static --disable-io-romio, ROMIO was no
longer adding some libraries to the wrapper LIBS list -- libraries
that just happened to also be needed by core OPAL/ORTE/OMPI layers.
The solution is not to use AC_CHECK_LIB or OPAL_CHECK_FUNC_LIB, but
use a pair of new macros:
* OPAL_SEARCH_LIBS_CORE: a wrapper around AC_SEARCH_LIBS. If we add
something to $LIBS, then also add it to the wrapper list of static
libraries. This is the main piece of functionality that was
wrong/missing.
* OPAL_SEARCH_LIBS_COMPONENT: similar to OPAL_SEARCH_LIBS_CORE, but
instead of directly adding it to the wrapper list of static
libaries, add it to <framework>_<component>_LIBS (which eventually
gets slurped up into the wrapper list of static libraries. See the
lengthy comment in config/opal_setup_wrappers.m4 near the beginning
of OPAL_SETUP_WRAPPER_INIT() for a more detailed explanation).
Most components did this correctly already, but one or two weren't
right, so I implemented this second macro quite similar to the
first and put it everywhere we already used AC_SEARCH_LIBS or
OPAL_CHECK_FUNC_LIB.
This needs to soak for a day or two on the trunk before moving to the
v1.8 branch.
Refs trac:4834
cmr=v1.8.2:reviewer=ggouaillardet
This commit was SVN r32447.
The following Trac tickets were found above:
Ticket 4834 --> https://svn.open-mpi.org/trac/ompi/ticket/4834
These messages were committed in the v1.8 branch in r32341, but were
never committed to the trunk (because we were waiting for the OPAL BTL
move). This commit brings the trunk and v1.8 help messages in line
with each other.
This commit was SVN r32445.
The following SVN revision numbers were found above:
r32341 --> open-mpi/ompi@5e752b4aba
Ensure that the connectivity checker agent only uses pointers from the
client that is the same process as the agent.
Not necessary for the v1.8 branch -- this is a trunk/v1.9-only problem.
This commit was SVN r32438.
Make the del_procs, module finalize, and endpoint destructors be the
same between trunk and v1.8, with one exception: the very beginning of
v1.8 module_finalize calls del_procs for each proc to simulate/pretend
the trunk/v1.9 PML behavior of calling del_procs before module_finalize.
This commit was SVN r32437.
Fixes an assertion failure in --enable-debug builds and SEGVs in normal
builds.
I'm not 100% sure I like this model, but it at least seems to be
consistent. Some variation on this scheme will need to be adapted to
the trunk, where usnic_del_procs() is called by the PML instead of
internally in usnic_finalize().
A related bug (but with different mechanics) is #4832.
This commit was SVN r32424.
Per patch from aryzhikh, initialize a copule of fields in the openib ini struct
Hard to understand why this has sat there for so long...
cmr=v1.8.2:reviewer=rhc:subject=initialize a copule of fields in the openib ini struct
This commit was SVN r32417.
The following Trac tickets were found above:
Ticket 4377 --> https://svn.open-mpi.org/trac/ompi/ticket/4377
Previously, the connectivity agent was pretty dumb: it took whatever
pings it got and ACKed them. Then we added an agent check to ensured
that the ping actually came from the source interface that it said it
came from. Now we add another check such that when a ping is received
on interface X that corresponds to usnic module Y, we ensure that the
source interface of the ping is on the all_endpoints list for module Y
(i.e., module Y expects to be able to talk to that peer interface).
This detects cases where peers have come to different conclusions
about which interfaces should be used to communicate (which is bad!).
This usually reflects a network misconfiguration.
Fixes CSCuq05389.
This commit was SVN r32383.
Ensure that incoming "ping" messages came from the IP address that
they think they came from. If they don't, drop them (because it is
probably routing error), which will likely eventually cause the
connectivity checker to timeout, and therefore cause the job to abort.
This commit was SVN r32368.
Update compat.h to only handle compatability between v1.7/v1.8 and
v1.9/2.0 (i.e., the current trunk). Remove what seems to be the last
vestiages of OMPI/ORTE pollution in the now-OPAL-ized usnic BTL.
Currently use a hard-coded constant for the MCW size (i.e.,
MPI_COMM_WORLD size) for some initialization values in the v1.9/2.0
series; still need to figure out something better there.
This commit was SVN r32365.
Adapt to moving down to OPAL: find a PML-registered error callback,
and use that when we don't have a module context and we need to abort.
Failing that, just call exit().
This commit was SVN r32362.
communication library should use to initialize itself.
Ralph will champion this change back with an RFC if there is a realistic
need/use case from the community.
This commit was SVN r32361.
The following SVN revision numbers were found above:
r32355 --> open-mpi/ompi@c903917f47
If --with-usnic is specified and we can't build the usnic BTL, abort.
If --without-usnic is specified, gracefully skip building the usnic
BTL. If neither is specified, do the OMPI-default behavior: try to
configure/build the usnic BTL, and if we can't, skip it.
Fixes CSCuq13889.
This commit was SVN r32349.
The only user of this code was coll/sm. I implemented a basic replacement
for the removed code. This gets the trunk compiling again with
--disable-dlopen.
This commit was SVN r32333.
common/ofacm is only used by the iboffload code in ompi. This code does
not currently work so it is safe to ignore these components until it is
fixed.
This commit was SVN r32331.
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
turned out to be a nightmare, as the pointers to the classes are
located in shared libraries memory areas, and are not accesible
after the shared library is unloaded. Thus, OPAL cannot cleanup
the left-over classes from the other shared libraries.
This commit was SVN r32248.
The following SVN revision numbers were found above:
r32236 --> open-mpi/ompi@59017433e1
new automake requires subdirs-object directive, to resolve this:
09:43:37 automake: warning: possible forward-incompatibility.
09:43:37 automake: At least a source file is in a subdirectory, but the 'subdir-objects'
09:43:37 automake: automake option hasn't been enabled. For now, the corresponding output
09:43:37 automake: object file(s) will be placed in the top-level directory. However,
09:43:37 automake: this behaviour will change in future Automake versions: they will
09:43:37 automake: unconditionally cause object files to be placed in the same subdirectory
09:43:37 automake: of the corresponding sources.
09:43:37 automake: You are advised to start using 'subdir-objects' option throughout your
09:43:37 automake: project, to avoid future incompatibilities.
09:43:37 tools/otfmerge/Makefile.common:13: warning: source file '$(OTFMERGESRCDIR)/otfmerge.c' is in a subdirectory,
09:43:37 tools/otfmerge/Makefile.common:13: but option 'subdir-objects' is disabled
cmr=v1.8.2:reviewer=ompi-rm1.8
This commit was SVN r32225.
new flag to ompi_info that allows a user to print all MCA variables of a specific type.
--type version_string
This command will print all MCA variables of type version_string.
This feature was developed by Elena Shipunova and was reviewed by Josh Ladd.
This commit was SVN r32166.
mpirun ... -x env_foo1=val1 -x env_foo2 -x env_foo3=val3 should now be expressed as
mpirun ... -mca mca_base_env_list env_foo1=val1+env_foo2+env_foo3=val3.
The motivation for doing this is so that a list of environment variables may be set via standard MCA mechanisms such as mca parameter files, amca lists, etc.
This feature was developed by Elena Shipunova and was reviewed by Josh Ladd.
This commit was SVN r32163.
After discussion with Nathan, change the ERR_VALUE_OUT_OF_BOUNDS to be
ERR_NOT_FOUND, for two reasons:
1. It's consistent with other uses of ERR_NOT_FOUND in the code.
1. In this case, we could have just looked up a variable that is
basically a "hole" -- e.g., var indexes 0, 1, 2 are valid, and 4,
5, and 5 are valid, but 3 is invalid (e.g., 3 was de-registered --
remember that MPI_T explicitly does not allow re-using indexes).
So returning ERR_OUT_OF_BOUNDS seems weird -- returning
ERR_NOT_FOUND seems a bit more natural.
cmr=v1.8.2:ticket=trac:4587
This commit was SVN r32158.
The following Trac tickets were found above:
Ticket 4587 --> https://svn.open-mpi.org/trac/ompi/ticket/4587
The alps ras and plm components were broken by recent changes in ORTE. This
commit resolves those issues.
Changes:
- Define PMI2_SUCCESS if it isn't defined. This fixes a problem with Cray's
PMI implementation which does not define (for some reason) PMI2_SUCCESS. We
had previously just used PMI_SUCCESS.
- Add missing definition and a typo in pml_alps_module.
- launch_id is no longer available in the orte_node_t structure. Use the
attribute lookup to get the value.
- Do not use an O(n^2) sorting algorithm when putting alps nodes in order. Use
opal_list_sort instead (O(nlogn)).
This commit was SVN r32076.
(a) default binding policy is in effect. In this case, we will emit a
warning and default to not binding unless the user provided the
"oversubscribe" or "overload" modifier to the "bind-to" option.
(b) user-specified binding policy is in effect. In this case, we will
error out unless the user provided the "oversubscribe" or "overload"
modifier to the "bind-to" option as we cannot meet the directive.
Either "bind-to" modifier (oversubscribe or overload) will be accepted for
now - in 1.9, we will deprecate the "overload" term in favor of
"oversubscribe".
Also added the ability to accept a --bind-to modifier without specifying the binding policy itself so a user can specify overload-allowed with the default policy.
Closes trac:4345
cmr=v1.8.2:reviewer=rhc:subject=resolve handling of overload conditions
This commit was SVN r32005.
The following Trac tickets were found above:
Ticket 4345 --> https://svn.open-mpi.org/trac/ompi/ticket/4345
* allow users to specify just a modifier for map-by instead of requiring that they also specify a policy. Thus, we now accept --map-by :pe=3 as indicating that we should use the default mapping policy, but bind 3 cpus/proc.
* if users specify a pe's/proc but no policy, default to --map-by NUMA to ensure we have access to multiple cpus for the request. This won't guarantee we have access to enough to meet the request, but gives us a chance. In addition, we know that binding a proc to multiple cpus will work best if those cpus are all in the same NUMA, so this provides some degree of optimized behavior.
Per a request from Jeff, define "oversubscribe" for binding as a synonym for the "overload" modifier.
cmr=v1.8.2:reviewer=rhc
This commit was SVN r31967.
http://www.open-mpi.org/community/lists/devel/2014/05/14822.php
Revamp the ORTE global data structures to reduce memory footprint and add new features. Add ability to control/set cpu frequency, though this can only be done if the sys admin has setup the system to support it (or you run as root).
This commit was SVN r31916.
during the unpack and during the positioning.
Fixes trac:4610.
This commit was SVN r31904.
The following Trac tickets were found above:
Ticket 4610 --> https://svn.open-mpi.org/trac/ompi/ticket/4610
Two leaks are fixed by this commit:
- opal_dss.lookup_data_type returns an allocated string. Free it.
- opal_ifaddrtokindex was leaking a struct addrinfo. Ensure that is
released before returning.
cmr=v1.8.2:reviewer=rhc
This commit was SVN r31777.
Either we need to ensure that all events are deleted (which may prove difficult), or find another way to release the base memory.
This commit was SVN r31775.
The following SVN revision numbers were found above:
r31765 --> open-mpi/ompi@75fb6dbba9
Need to remove the items of the list to avoid an assert in debug builds.
cmr=v1.8.2:ticket=trac:4628
This commit was SVN r31769.
The following SVN revision numbers were found above:
r31764 --> open-mpi/ompi@13fd6ae774
The following Trac tickets were found above:
Ticket 4628 --> https://svn.open-mpi.org/trac/ompi/ticket/4628
This fixes a series of leaks identified by valgrind. Since
opal_event_base_open was creating the event base it follows that
opal_event_base_close should release the event base.
cmr=v1.8.2:reviewer=rhc
This commit was SVN r31765.
list
This commit updates the behavior of opal_free_list_t to match the
behavior of ompi_free_list_t. opal_free_list_t constructed items
placed on the free list but never destructed them.
cmr=v1.8.2:reviewer=jsquyres
This commit was SVN r31764.
Bring down 3aa0ed6 from the hwloc v1.7 branch: Stevens says we should
GETFD before we SETFD, so we do
cmr=v1.8.2:reviewer=rhc
This commit was SVN r31683.
top_ompi_srcdir -> OMPI_TOP_SRCDIR
top_ompi_builddir -> OMPI_TOP_BUILDDIR
We also split the srcdir/builddir flags according to their local tree (e.g., OPAL_TOP_SRCDIR), and tied them all together in configure.ac. Renamed ompi_ignore and ompi_unignore to be opal_<foo> as these are agnostic markers.
Only thing left is ompilibdir being treated similar to what we dif for srcdir/builddir. Coming soon.
This commit was SVN r31678.
- This line, and those below, will be ignored--
M opal/mca/event/libevent2021/configure.m4
M opal/mca/hwloc/hwloc172/configure.m4
M configure.ac
M config/opal_setup_libltdl.m4
M config/opal_check_visibility.m4
M config/opal_setup_cc.m4
This commit was SVN r31637.
have any impact on the type signature.
Fixes trac:4597.
cmr=v1.8.2:reviewer=ompi-rm1.8
This commit was SVN r31635.
The following Trac tickets were found above:
Ticket 4597 --> https://svn.open-mpi.org/trac/ompi/ticket/4597
Several fixes:
- I was allowing an MPI_T_cvar_handle to be created for an invalid
variable. Fixed this by checking if the variable is valid in
mca_base_var_get.
- Use a better error code when the caller tries to create an unbound
pvar handle for a bound variable.
- Return the verbosity level in MPI_T_cvar_get_info.
cmr=v1.8.2:reviewer=jsquyres
This commit was SVN r31576.
http://www.open-mpi.org/community/lists/devel/2014/04/14496.php
Revamp the opal database framework, including renaming it to "dstore" to reflect that it isn't a "database". Move the "db" framework to ORTE for now, soon to move to ORCM
This commit was SVN r31557.
code.
This commit fixes minor errors in the incorrectly-committed r31513
(new fd close-on-exec convenience function).
Refs trac:4550
This commit was SVN r31514.
The following SVN revision numbers were found above:
r31513 --> open-mpi/ompi@e1655ae68d
The following Trac tickets were found above:
Ticket 4550 --> https://svn.open-mpi.org/trac/ompi/ticket/4550
Paul Hargrove pointed out that Stevens tells us that we should
FD_GETFL before FD_SETFL. And so we shall.
Make a new convenience function to do this (opal_fd_set_cloexec()),
just so that we don't have to litter this 2-step process throughout
the code.
Refs trac:4550
This commit was SVN r31513.
The following Trac tickets were found above:
Ticket 4550 --> https://svn.open-mpi.org/trac/ompi/ticket/4550
Make sure that an internal, long-lived hwloc fd is marked as
close-on-exec so that children don't inherit it. This patch is
committed upstream in the hwloc master and v1.9 branches as 7489287
and b654e19, respectively. The patch applied here is the exact same
logic, but the surrounding code changed slightly since the hwloc v1.7
series, so the patch doesn't apply cleanly.
Refs trac:4550
This commit was SVN r31511.
The following Trac tickets were found above:
Ticket 4550 --> https://svn.open-mpi.org/trac/ompi/ticket/4550
completed. As we need to have the right displacement at the beginning
of the next call, we should save the position relative to the beginning
of the buffer and not to the last loop.
This commit was SVN r31387.
Aside from comments, these help files are empty -- so just remove
them. Also remove a few inclusions of show_help.h, since it isn't
used in those source files.
This commit was SVN r31328.
The contents of this file were superceded by mca-help-var.txt.
Indeed, this file was already removed from Makefile.am; it looks like
it was accidentally left in SVN.
This commit was SVN r31327.
Also:
* fix formatting error at top (make the emacs format line begin
with a comment character)
* Remove blank lines between topics and replace them with empty
comment lines (so that the help messages are not displayed with a
trailing blank line)
This commit was SVN r31324.
Also, add missing ORTE_ERROR_LOG in the other case where this error
message is used (i.e., ORTE_ERROR_LOG was used in the one place, so
let's also use it in the other place).
This commit was SVN r31321.
This type of help message is now handled by the MCA var system itself
(via the MCA_BASE_VAR_FLAG_ENVIRONMENT_ONLY flag at var_register
time).
This commit was SVN r31318.
add -mca base_env_list "var1=val1 var2=val2 ..." mca parameter that can be used in mca param files
or with -am app.conf mpirun commandline to set rank env variables with mca mechanism
fixed by Elena, reviewed by Miked
cmr=v1.8.1:reviewer=ompi-rm1.8
This commit was SVN r31302.
Also, since I put some of the macros for these silent/verbose rules up
in the top-level Makefile.man-page-rules file, I renamed it to
Makefile.ompi-rules.
I've had this sitting around for a while; now seems like as good a
time as any to commit it.
This commit was SVN r31271.
The semantics of the variable mpi_yield_when_idle are to call
opal_progress_set_yield_when_idle at MPI_Init. It would be difficult
to modify the old variable to support setting this parameter at
runtime. The fix is to add an additional parameter to opal:
opal_progress_yield_when_idle that directly sets the variable. This
variable is settable anytime and does not affect the semantics of
the old mpi_yield_when_idle variable.
Refs trac:193
cmr=v1.8.1:reviewer=jsquyres
This commit was SVN r31255.
The following Trac tickets were found above:
Ticket 193 --> https://svn.open-mpi.org/trac/ompi/ticket/193
on some linux distro (sles11sp2) csh fails to parse $LS_COLORS and borks with error:
Unknown colorls variable `mh'.
The workaround is to unset LS_COLORS before calling to csh script
reviewed by Jeff
cmr=v1.8:reviewer=ompi-rm1.8
This commit was SVN r31244.
opal-restart.c disables crs module selection by setting
crs_base_do_not_select to true using opal_setenv() before
opal_init(). After opal_init() it enables module selection
by changing the variable back to false using opal_setenv().
This does not work anymore as the variables are only read
from the environment during variable registration.
This changes the second opal_setenv() to mca_base_var_set_value()
This commit was SVN r31108.
an enumerator's mca_base_var_enum_sfv_fn_t can be NULL.
cmr=v1.7.5:ticket=trac:4398:reviewer=ompi-gk1.7
This commit was SVN r31085.
The following Trac tickets were found above:
Ticket 4398 --> https://svn.open-mpi.org/trac/ompi/ticket/4398
In the OPAL_ENABLE_FT_CR code path there used to be a variable
'mca_base_component_distill_checkpoint_ready' which got removed.
The FT code was not compiling and while trying to get it to compile
again the old variable was #ifdef'd out. This re-introduces the
variable with a new name 'opal_base_distill_checkpoint_ready'
and enables the code previously #ifdef'd out.
This removes the last hack introduced to get the FT code to compile
again.
This commit was SVN r30928.
* Include opal_stdint.h so that we have uin32_t
cmr=v1.7.5:ticket=trac:4298
This commit was SVN r30890.
The following Trac tickets were found above:
Ticket 4298 --> https://svn.open-mpi.org/trac/ompi/ticket/4298
1. Changed rng_buff_t --> opal_rng_buff_t
2. All global variables obey the prefix rule
3. Old code has been removed
4. Found a couple of unnecessary includes
Refs trac:4298
This commit was SVN r30807.
The following Trac tickets were found above:
Ticket 4298 --> https://svn.open-mpi.org/trac/ompi/ticket/4298
This adds the code to actually checkpoint a process using CRIU
with the necessary variables to control the behaviour.
Right now only --np 1 is supported and --mca oob tcp.
Following parameters are supported:
* crs_criu_log: name of the log file
* crs_criu_log_level: verbosity level in the log file
* crs_criu_tcp_established: C/R established TCP connections
* crs_criu_shell_job: C/R shell jobs
* crs_criu_ext_unix_sk: allow external unix connections
* crs_criu_leave_running: leave tasks in running state after checkpoint
This commit was SVN r30772.
* Remove redundant/unnecessary uses of $2
* Change a bunch of logic from negative to positive
* Use OPAL_VAR_SCOPE_PUSH/POP to help reduce env var usage
* Only use "" in test statements with strings that require sanitization
* Removed redundant AC_MSG_WARN/ERROR. There's now only one check at
the bottom for whether the component is "good" or not. We'll
AC_MSG_WARN/ERROR in that one location.
Thanks to Jeff Squyres for this patch.
This commit was SVN r30739.
To be able to checkpoint/restart using criu (criu.org) a new
CRS component is added which is based on criu. This first commit
provides the minimal set of functions and configure script options
to enable --with-criu and link against libcriu.so.
No actual checkpoint/restart functionality is yet implemented.
This is only the framework which needs to be filled with the
actual functionality.
This commit was SVN r30666.
Wire the security check into ORTE's OOB handshake, and add a "version" check to ensure that both ends are from the same ORTE version. If not, report the mismatch and refuse the connection
Fixes trac:4171
cmr=v1.7.5:reviewer=jsquyres:subject=Add a security framework for authenticating connections
This commit was SVN r30551.
The following Trac tickets were found above:
Ticket 4171 --> https://svn.open-mpi.org/trac/ompi/ticket/4171
Since Paul is the only one of the team with the required hardware to test it, and he has done so, consider this RM-approved.
cmr=v1.7.5:reviewer=ompi-gk1.7
This commit was SVN r30401.
Thanks to Paul Hargrove for tracking it down.
RM-approved
cmr=v1.7.4:reviewer=ompi-gk1.7
This commit was SVN r30397.
The following SVN revision numbers were found above:
r26226 --> open-mpi/ompi@12781482b9
Refs trac:4117. Please use this commit rather than the patch attached to
the ticket; the patch had a few mistakes in the tweaked wording.
This commit was SVN r30362.
The following SVN revision numbers were found above:
r30298 --> open-mpi/ompi@58479399c3
The following Trac tickets were found above:
Ticket 4117 --> https://svn.open-mpi.org/trac/ompi/ticket/4117
work around buggy NUMA node cpusets (i.e., buggy BIOSs).
Thanks to Jeff Becker for reporting the issue.
Submitted by Brice Goglin, reviewed by Jeff Squyres.
cmr=v1.7.4:reviewer=ompi-rm1.7
This commit was SVN r30306.
The original code was passing a union by value, and doing odd things
on Solaris/SPARC (where "odd" rhymes with "SIGBUS"). Replace it with
an exploded switch/case block for all the enum values. Also use the
string literals so that we get compiler checking of the format string
vs. the type of the actual arguments.
cmr=v1.7.4:revier=hjelmn:subject=Fix MCA base var to not pass union by value
This commit was SVN r30276.
variable names for deprecated variables.
Closes trac:3270
cmr=v1.7.4:reviewer=jsquyres
This commit was SVN r30275.
The following Trac tickets were found above:
Ticket 3270 --> https://svn.open-mpi.org/trac/ompi/ticket/3270
* Fix some typos in macro names.
* Add case for OS's that have statfs() but no struct statfs (!).
* Add case for NetBSD with struct statvfs.f_fstypename.
Many thanks to Paul Hargrove who developed the majority of this patch.
Reviewed by Jeff Squyres.
cmr=v1.7.4:reviewer=ompi-rm1.7
This commit was SVN r30255.
Change the logic in bind.c to only include <malloc.h> if we don't have posix_memalign.
In http://www.open-mpi.org/community/lists/devel/2014/01/13619.php,
Paul Hargrove found a compiler warning on OpenBSD where <malloc.h>
exists, but is not intended to be used (and doesn't error out, so
AC_CHECK_HEADERS says its ok).
Reviewed by Brice Goglin.
cmr=v1.7.4:reviewer=ompi-rm1.7
This commit was SVN r30234.
As noted by Paul Hargrove, the #if's surrounding the use of statfs()
and statvfs() in opal/util/path.c have apparently gotten stale (e.g.,
modern flavors of *BSD OSs no longer define __BSD). Changes:
* Add statfs and statvfs to the AC_CHECK_FUNCS in configure.ac
* Add a sanity check to ensure that we have at least one of statfs()
or statvfs(). Add a similar sanity check in opal/util/path.c, just
as defensive programming.
* Use AC_CHECK_MEMBERS in configure.ac to check for specific struct
statfs/struct statvfs members that we use in opal/util/path.c
* In path.c, add some #includes as listed on the OS man page for
statfs(2) (OS X 10.8.5/Mountain Lion)
* The previous code used statvfs() on Solaris and statfs() everywhere
else. Attempting to replicate this with behavior-based configure
testing led to fairly complicted if/else logic, so the new code
uses whichever of the two are available (i.e., it might actually
use both -- OS X 10.8.5 and RHEL 6.5 have both statfs() and
statvfs()). The rationale here is that we don't really care which
of the two functions report the answer; we'll take the answer
regardless of where it comes from. For example, if one function
returns a failure and the other does not, we'll use the results
from the successful function and ignore the failed one.
This new code seems to work on OS X and Linux. We'll have to see what
happens with MTT and future Paul Hargrove testing...
cmr=v1.7.4:reviewer=ompi-rm1.7:subject=Make statfs/statvfs more robust
This commit was SVN r30198.
configury/Makefile.am changes; this commit renames the internal
installdirs.h framework struct field names to match the configry macro
names:
* pkgdatdir -> ompidatadir
* pkglibdir -> ompilibdir
* pkgincludedir -> ompiincludedir
This commit was SVN r30145.
The following SVN revision numbers were found above:
r30140 --> open-mpi/ompi@8b778903d8
pkg{data,lib,includedir}, use our own ompi{data,lib,includedir}, which is
always set to {datadir,libdir,includedir}/openmpi. This will keep us from
having help files in prefix/share/open-rte when building without Open MPI,
but in prefix/share/openmpi when building with Open MPI.
This commit was SVN r30140.
Right now the C/R code fails because of a change introduced in
opal/mca/compress/base/compress_base_open.c and
pal/mca/crs/base/crs_base_open.c in 2013 with commit
git 734c724ff76d9bf814f3ab0396bcd9ee6fddcd1b
svn r28239
Update OPAL frameworks to use the MCA framework system.
This commit changed a lot but also the return value of functions from
OPAL_SUCCESS to OPAL_ERR_NOT_AVAILABLE/OPAL_ERR_NOT_AVAILABLE.
This commit lets opal_compress_base_register() and opal_crs_base_open()
always return OPAL_SUCCESS and removes unneeded #includes.
This commit was SVN r30130.
The following SVN revision numbers were found above:
r28239 --> open-mpi/ompi@365cf48db5
r30086: make sure that a super item is constructed properly).
Refs trac:4035
This commit was SVN r30090.
The following SVN revision numbers were found above:
r30086 --> open-mpi/ompi@d1c63f878e
The following Trac tickets were found above:
Ticket 4035 --> https://svn.open-mpi.org/trac/ompi/ticket/4035
Thanks to Tetsuya Mishima for identifying the problem and providing the patch!
cmr=v1.7.4:reviewer=jsquyres:subject=Fix LAMA mapper for PGI compilers
This commit was SVN r30086.
Fix comm_spawn on a single host - with the new default mapping scheme, we were incorrectly computing the number of procs to put on the node.
Refs trac:4003
This commit was SVN r30033.
The following Trac tickets were found above:
Ticket 4003 --> https://svn.open-mpi.org/trac/ompi/ticket/4003
- Use ->boolval for booleans when creating a string.
- Solaris has some issue with the ?: used in one of find functions. Use an if instead.
- Change all instances of index -> vari to avoid issues with redefining index.
cmr=v1.7.4:reviewer=jsquyres
This commit was SVN r29997.
atomics in the critical path and are not currently used. We can bring them back if there
turns out to be a good use for them.
cmr=v1.7.4:reviewer=brbarret
This commit was SVN r29994.
added on top of r29991 and:
* Consolidates the _debug variables in opal_datatype_internal.h and
opal_convertor.h
* Puts the DO_DEBUG macros back in the .c files, because they are
slightly different from each other
Refs trac:4004
This commit was SVN r29992.
The following SVN revision numbers were found above:
r29991 --> open-mpi/ompi@a88e143127
The following Trac tickets were found above:
Ticket 4004 --> https://svn.open-mpi.org/trac/ompi/ticket/4004
and extern'ed s an int in another. This caused a SIBGUS on
Solaris/SPARC.
This commit properly moves the extern to a .h file so that it's the
same in all files. It also moves the DO_DEBUG to the header file,
because it was defined to the same thing in multiple .c files.
cmr=v1.7.4:reviewer=bosilca:subject=fix SPARC SIGBUS in opal convertor code
This commit was SVN r29991.
The following Trac tickets were found above:
Ticket 3989 --> https://svn.open-mpi.org/trac/ompi/ticket/3989
and preventing access to potentially unaligned data.
Reviewed by Dave Goodell. Tested by Siegmarr Gross.
cmr=v1.7.4:reviewer=ompi-rm1.7:subject=fix SPARC SIGBUS in opal net code
This commit was SVN r29983.
The following Trac tickets were found above:
Ticket 3990 --> https://svn.open-mpi.org/trac/ompi/ticket/3990
Reset topology usage for each node as we bind as multiple nodes may be linked to the same topology object. This will need to be revisited for scale as it does take some non-zero time to reset the usage each iteration. However, storing individual topology objects for every node consumes memory, so it's a tradeoff.
cmr=v1.7.4:reviewer=jsquyres:subject=Eliminate excessive binding/memory warnings
This commit was SVN r29978.
compiled with ummunotify support (which is the check that r29720 just
recently added).
This commit was SVN r29961.
The following SVN revision numbers were found above:
r29720 --> open-mpi/ompi@ae8c826527
includes various fixes all over the C/R code which are
hard to group like the other patches.
Changes from V1:
* explain why mca_base_component_distill_checkpoint_ready no longer works
* compare return result of opal functions with OPAL_* values
Changes from V2:
* use orte_rml_oob_ft_event() instead of referencing through the modules
* properly protect variable (thanks to --enable-picky)
This commit was SVN r29922.
should be already set to the right value. This fixes a problem
identified by Guillaume Gouaillardet, where using a single
persistent receive leads to leaking the convertor stack memory.
Refs trac:3956
cmr=v1.7.4:reviewer=jsquyres:subject=Correctly handle the convertor internal stack for persistent receives.
This commit was SVN r29920.
The following Trac tickets were found above:
Ticket 3956 --> https://svn.open-mpi.org/trac/ompi/ticket/3956
* default to bind-to core
* map-by slot if np=2
* map-by socket (balance across sockets on each node) if np > 2
* map-by <obj> will imply rank-by <obj> by default (leave default binding as above)
Fix a bug in the map-by <obj> mapper where we incorrectly compute the #procs to assign if the #slots > #procs
cmr=v1.7.4:reviewer=jsquyres:subject=Update default binding and mapping values
This commit was SVN r29919.
got linked together (work on one caused work in the other):
* Clean up a bunch of VAR_SCOPE issues in configure. This includes:
* Using VAR_SCOPE_PUSH and VAR_SCOPE_POP in more places
* Cleaning up the use of some shell variables (e.g., name them better)
* Add support for external libevent via
--with-libevent=<dir-to-libevent-install-tree>, as specifically
asked for by downstream packagers.
* Revamp how wrapper compiler RPATH (and RUNPATH) support is done.
The external libevent work exposed weakenesses in how the original
RPATH/RUNPATH work was done, so we had to re-do it to be a bit more
robust.
This work has not yet been tested on Solaris.
Refs trac:3694
This commit was SVN r29899.
The following Trac tickets were found above:
Ticket 3694 --> https://svn.open-mpi.org/trac/ompi/ticket/3694
more:
- Remove OPAL_ENABLE_MULTI_THREADS, since it didn't really do anything
correctly. Opal always has threads enabled at this point.
- Remove OMPI_ENABLE_PROGRESS_THREADS, since this hasn't worked in
8 years and it has performance issues we'll never be able to
overcome. Note that we have plans for re-adding async progress, using
a hybrid protocol of async and sync sends.
- OMPI_ENABLE_THREAD_MULTIPLE now determines whether the thread lock
macros do the check or not.
- Condition variables are ALWAYS polling right now, which fixes the thread
live-lock currently found when THREAD_MULTIPLE is turned on.
This commit was SVN r29891.
This is helpful in the work for #3694: ensure that many places that
eventually end up in configure don't overly-pollute the global shell
variable space (because debugging accidental shell variable pollution
can be a real pain).
Refs trac:3694
This commit was SVN r29830.
The following Trac tickets were found above:
Ticket 3694 --> https://svn.open-mpi.org/trac/ompi/ticket/3694
* Ensure "cnt" is always initialized
* Ensure we dont' buffer overflow on strncat() -- need to ensure we
account for the terminating \0 character
* hwloc_get_type_depth() returns an int (not unsigned), and
HWLOC_TYPE_DEPTH_UNKNOWN if it's unknown (which is probably <0, but
still, might as well check what the official hwloc docs say to
check for)
cmr=v1.7.4:reviewer=rhc:subject=fix hwloc base compiler warnings
This commit was SVN r29686.
Change static opal_setlimit() function to return its value in an OUT
parameter and return the usual int error code indicating success or
failure.
The OUT param and return code need to be separated because the OUT
param is an unsigned type, but opal_setlimit() was returning -1 upon
failure. Hence, the caller could not know that it had failed because
the return type was previously an unsigned type.
cmr=v1.7.4:reviewer=rhc:subject=Fix opal sys_limits.c signed/unsigned warnings
This commit was SVN r29685.
should have been all along and fix one place that uses the file
Update opal_portable_platform.h with changes to mpi_portable_platform.h made
in r29608.
Make mpi_portable_platform.h a symlink to opal_portable_platform.h, so that
they won't get out of sync. I'd like to remove mpi_portable_platform.h, but
we don't automatically add -I${includedir}/openmpi/ to make that sane from
a header include point of view, so that's future work.
This commit was SVN r29618.
The following SVN revision numbers were found above:
r29608 --> open-mpi/ompi@b71bd51cdd
this change the next time we update libevent.
cmr=v1.7.4:ticket=3882
This commit was SVN r29597.
The following Trac tickets were found above:
Ticket 3882 --> https://svn.open-mpi.org/trac/ompi/ticket/3882
OSX atomic support is disabled by default. Enable with --enable-osx-builtin-atomics.
Fixes trac:2120
This commit was SVN r29568.
The following Trac tickets were found above:
Ticket 2120 --> https://svn.open-mpi.org/trac/ompi/ticket/2120
- Make a copy of enumerator data for default enumerators. This will allow
the caller to free their data once the enumerator has been created. This
is a change from just referencing the values array.
- Make mca_base_pvar_notify check if the pvar is valid before calling the
notify callback. This fixes a segmentation fault when destroying handles
after MPI_Finalize().
cmr=v1.7.4:ticket=trac:3861
This commit was SVN r29512.
The following Trac tickets were found above:
Ticket 3861 --> https://svn.open-mpi.org/trac/ompi/ticket/3861
First cut does not attempt any "cross-check". As we discover compilers
which complain about __noinline__, we will add specific cross checks to
handle those cases.
Reviewed-by: Jeff Squyres <jsquyres@cisco.com>
This commit was SVN r29488.
Fixes:
- Segmentation fault when using watermark variables.
- Segmentation fault when using a handle bound to a no longer valid
performance variable.
- Incorrect return codes from MPI_T_pvar_* functions.
cmr=v1.7.4:reviewer=jsquyres
This commit was SVN r29481.
This change contains a non-mandatory modification
of the MPI-RTE interface. Anyone wishing to support
coprocessors such as the Xeon Phi may wish to add
the required definition and underlying support
****************************************************************
Add locality support for coprocessors such as the Intel Xeon Phi.
Detecting that we are on a coprocessor inside of a host node isn't straightforward. There are no good "hooks" provided for programmatically detecting that "we are on a coprocessor running its own OS", and the ORTE daemon just thinks it is on another node. However, in order to properly use the Phi's public interface for MPI transport, it is necessary that the daemon detect that it is colocated with procs on the host.
So we have to split the locality to separately record "on the same host" vs "on the same board". We already have the board-level locality flag, but not quite enough flexibility to handle this use-case. Thus, do the following:
1. add OPAL_PROC_ON_HOST flag to indicate we share a host, but not necessarily the same board
2. modify OPAL_PROC_ON_NODE to indicate we share both a host AND the same board. Note that we have to modify the OPAL_PROC_ON_LOCAL_NODE macro to explicitly check both conditions
3. add support in opal/mca/hwloc/base/hwloc_base_util.c for the host to check for coprocessors, and for daemons to check to see if they are on a coprocessor. The former is done via hwloc, but support for the latter is not yet provided by hwloc. So the code for detecting we are on a coprocessor currently is Xeon Phi specific - hopefully, we will find more generic methods in the future.
4. modify the orted and the hnp startup so they check for coprocessors and to see if they are on a coprocessor, and have the orteds pass that info back in their callback message. Automatically detect that coprocessors have been found and identify which coprocessors are on which hosts. Note that this algo isn't scalable at the moment - this will hopefully be improved over time.
5. modify the ompi proc locality detection function to look for coprocessor host info IF the OMPI_RTE_HOST_ID database key has been defined. RTE's that choose not to provide this support do not have to do anything - the associated code will simply be ignored.
6. include some cleanup of the hwloc open/close code so it conforms to how we did things in other frameworks (e.g., having a single "frame" file instead of open/close). Also, fix the locality flags - e.g., being on the same node means you must also be on the same cluster/cu, so ensure those flags are also set.
cmr:v1.7.4:reviewer=hjelmn
This commit was SVN r29435.
Fix two problems that surfaced when using direct launch under SLURM:
1. locally store our own data because some BTLs want to retrieve
it during add_procs rather than use what they have internally
2. cleanup MPI_Abort so it correctly passes the error status all
the way down to the actual exit. When someone implemented the
"abort_peers" API, they left out the error status. So we lost
it at that point and *always* exited with a status of 1. This
forces a change to the API to include the status.
cmr:v1.7.3:reviewer=jsquyres:subject=Fix MPI_Abort and modex_recv for direct launch
This commit was SVN r29405.
