Bring Slurm PMI-1 component online
Bring the s2 component online
Little cleanup - let the various PMIx modules set the process name during init, and then just raise it up to the ORTE level. Required as the different PMI environments all pass the jobid in different ways.
Bring the OMPI pubsub/pmi component online
Get comm_spawn working again
Ensure we always provide a cpuset, even if it is NULL
pmix/cray: adjust cray pmix component for pmix
Make changes so cray pmix can work within the integrated
ompi/pmix framework.
Bring singletons back online. Implement the comm_spawn operation using pmix - not tested yet
Cleanup comm_spawn - procs now starting, error in connect_accept
Complete integration
This commit adds implementations for opal_atomic_lifo_pop and
opal_atomic_lifo_push that make use of the load-linked and
store-conditional instruction. These instruction allow for a more
efficient implementation on supported platforms.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit does two things. It removes checks for C99 required
headers (stdlib.h, string.h, signal.h, etc). Additionally it removes
definitions for required C99 types (intptr_t, int64_t, int32_t, etc).
Signed-off-by: Nathan Hjelm <hjelmn@me.com>
This commit enables the use of opal classes after a call to
opal_class_finalize. This is needed to support re-initializing opal
after calling opal_finalize_util (). This is needed to support the
following without leaking:
MPI_T_init_thread ();
MPI_T_finalize ();
MPI_Init ();
MPI_Finalize ();
Before this commit the above code would crash in MPI_Init because the
constructor array for some class was freed by opal_class_finalize ().
The fix is to turn the cls_initialized member of opal_class_t into an
init epoch identifier and compare it against opal_class_init_epoch
instead of 1. On each call to opal_class_finalize the
opal_class_init_epoch counter is incremented forcing re-initialization
of classes after finalize.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Use of the old ompi_free_list_t and ompi_free_list_item_t is
deprecated. These classes will be removed in a future commit.
This commit updates the entire code base to use opal_free_list_t and
opal_free_list_item_t.
Notes:
OMPI_FREE_LIST_*_MT -> opal_free_list_* (uses opal_using_threads ())
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Historically these two lists were different due to ompi_free_list_t
dependencies in ompi (mpool). Those dependencies have since been moved
to opal so it is safe to (finally) combine them. The combined free
list comes in three flavors:
- Single-threaded. Only to be used when it is guaranteed that no
concurrent access will be made to the free list. Single-threaded
functions are suffixed with _st.
- Mutli-threaded. To be used when the free list may be accessed by
multiple threads despite the setting of opal_using_threads.
Multi-threaded functins are suffixed with _mt.
- Conditionally multi-threaded. Common use case. These functions are
thread-safe if opal_using_threads is set to true.
Compatibility functions for the ompi_free_list_t and the old accessor
functions (OPAL_FREE_LIST_*) are available while the code base is
transitioned to the new class/functions.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
On x86_64 reading a 128-bit value requires multiple instructions.
Under some conditions if the counted pointer counter is read before
the item pointer the fifo can be left in an inconsistent state. This
commit forces the read of the counter to always be read first.
The fifo does not appear to suffer from the same race.
It is possible the compiler can reorder the read of the head item and
the head itself. This could lead to a situation where the item
returned was not really the head item.
This commit adds a new class: opal_fifo.h. The new class has atomic, non-atomic,
and opal_using_threads() conditoned routines. It should be used when first-in
first-out is required and should perform much better than using locks and an
opal_list_t. Like with opal_lifo_t there are two versions of the atomic
implementation: 128-bit compare-and-swap, and spin-locked. More implementations
can be added later (LL/SC comes to mind).
This commit also adds a unit test for the opal_fifo_t class. This test verifies
the fifo implementation when using multiple threads.
- Rename opal_atomic_lifo_t to opal_lifo_t to reflect both atomic and
non-atomic usage. Added new routines (opal_lifo_*_st) for non-atomic
usage as well as routines conditioned off opal_using_threads(). The
atomic versions are always thread safe and the non-atomic are always
not thread safe.
- Add a new atomic lifo implementation that makes use of 128-bit
compare-and-swap. The new implementation should scale better with
larger numbers of threads.
- Add threading unit test for opal_lifo_t.
We recognize that this means other users of OPAL will need to "wrap" the opal_process_name_t if they desire to abstract it in some fashion. This is regrettable, and we are looking at possible alternatives that might mitigate that requirement. Meantime, however, we have to put the needs of the OMPI community first, and are taking this step to restore hetero and SPARC support.
These two macros set the prefix for the OPAL and ORTE libraries,
respectively. Specifically, the OPAL library will be named
libPREFIXopen-pal.la and the ORTE library will be named
libPREFIXopen-rte.la.
These macros must be called, even if the prefix argument is empty.
The intent is that Open MPI will call these macros with an empty
prefix, but other projects (such as ORCM) will call these macros with
a non-empty prefix. For example, ORCM libraries can be named
liborcm-open-pal.la and liborcm-open-rte.la.
This scheme is necessary to allow running Open MPI applications under
systems that use their own versions of ORTE and OPAL. For example,
when running MPI applications under ORTE, if the ORTE and OPAL
libraries between OMPI and ORCM are not identical (which, because they
are released at different times, are likely to be different), we need
to ensure that the OMPI applications link against their ORTE and OPAL
libraries, but the ORCM executables link against their ORTE and OPAL
libraries.
the OPAL and ORTE libraries. This is required by projects such as ORCM
that have their own ORTE and OPAL libraries in order to avoid library
confusion. By renaming their version of the libraries, the OMPI
applications can correctly dynamically load the correct one for their
build."
This reverts commit 63f619f8719fb853d76130d667f228b0a523bd60.
Revert "OPAL: drop dead with core on bad flow. rarely happens with helloworld on large scale."
This reverts commit 86f1d5af3ee484f34092ad3f7a645d9a5ccbcb6c.
Will be reconsidered via RFC as it represents a significant change in behavior
1. It's actually hashing now, whereas the old OPAL hash table was not. Thus, it is a bug fix for and, as such, should be included in the 1.8 series.
2. It is dynamic and can grow and shrink the number of buckets in accordance with job size, whereas the old OPAL hash table had a fixed number of buckets which resulted in poor retrieval performance at large scale.
This scheme has been deployed in the field on very large H.P./Mellanox systems and has been demonstrated to significantly decrease job start-up time (~ 20% improvement) when launching applications directly with srun in SLURM environments. However, neither SLURM nor direct launch are prerequisites to take advantage of this change as any entity that utilizes OPAL hash table objects can benefit (at least partially) from this contribution.
With --enable-memchecker builds, use calloc(3) for OBJ_NEW instead of
malloc(3). This cuts down on a lot of valgrind/memory checker false
positive output.
Also make a minor change in the valgrind configure.m4; have it assign
0xf to a char. The prior assignment (of 0xff) was warning about an
overflow. This didn't really matter, but we might as well make the
test not have a gratuitious warning in it.
Decrease the hash table size when an element is removed
cmr=v1.8.2:ticket=trac:4872
This commit was SVN r32566.
The following Trac tickets were found above:
Ticket 4872 --> https://svn.open-mpi.org/trac/ompi/ticket/4872
use HAVE_STRINGS_H to protect <strings.h> include
cmr=v1.8.2:reviewer=rhc:ticket=trac:4853
This commit was SVN r32510.
The following Trac tickets were found above:
Ticket 4853 --> https://svn.open-mpi.org/trac/ompi/ticket/4853
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
