Before dynamic add_procs support was committed to master we called
add_procs with every proc in the job. The XRC code in the openib btl
was taking advantage of this and setting the number of work queue
entries (WQE) based on all the procs on a remote node. Since that is
no longer the case we can not simply increment the sd_wqe field on the
queue pair. To fix the issue a new field has been added to the xrc
queue pair structure to keep track of how many wqes there are total on
the queue pair. If a new endpoint is added that increases the number
of wqes and the xrc queue pair is already connected the code will
attempt to modify the number of wqes on the queue pair. A failure is
ignored because all that will happen is the number of active send work
requests on an XRC queue pair will be more limited.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit reduces the overhead of calling the ugni progress
function. It does the following:
- Check for new connections once every eight calls.
- Do not call remote smsg progress unless we are connected to at
least one remote peer.
- Do not call rdma progress unless at least one rdma fragment is
outstanding.
- Check endpoint wait list size before obtaining a lock.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit reduces the default exclusivity so that btl/scif is not
used for send/recv over other shared memory transports.
Fixesopen-mpi/ompi#1712
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Our checks and the ones of libevent are somewhat flawed.
If adding multiple "-framework" to CXXFLAGS or CFLAGS, we strip
the keyword from the command-line, not good.
libevent however assumes plain gcc without testing properly
that the compiler supports -Wno-deprecated-declarations.
* Remodel the request.
Added the wait sync primitive and integrate it into the PML and MTL
infrastructure. The multi-threaded requests are now significantly
less heavy and less noisy (only the threads associated with completed
requests are signaled).
* Fix the condition to release the request.
This commit changes the behavior of bml/r2 from conditionally
registering btl progress functions to always registering progress
functions. Any progress function beloning to a btl that is not yet in
use is registered as low-priority. As soon as a proc is added that
will make use of the btl is is re-registered normally.
This works around an issue with some btls. In order to progress a
first message from an unknown peer both ugni and openib need to have
their progress functions called. If either btl is not in use after the
first call to add_procs the callback was never happening. This commit
ensures the btl progress function is called at some point but the
number of progress callbacks is reduced from normal to ensure lower
overhead when a btl is not used. The current ratio is 1 low priority
progress callback for every 8 calls to opal_progress().
Fixesopen-mpi/ompi#1676
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Take another shot at untangling the spaghetti
orterun: fix for command line parsing
orte-submit calls opal_init_util () before parsing out MCA command line
options (-mca, -am, etc). This prevents mpirun from setting opal MCA
variables for some frameworks as well as the MCA base. This is because
when a framework is opened all of its variables are set to read-only.
Eventually we want to lift this restriction on some MCA variables but
since -mca is affected we must parse out the MCA command line options
before opal_init_util(). This commit fixes the bug by adding a new
option to opal_cmd_line_parse (ignore unknown option) so orte-submit
can pre-parse the command line for MCA options.
Signed-off-by: Nathan Hjelm <hjelmn@me.com>
Minor cleanups to avoid releasing/recreating the cmd line
Remove contrib/windows/
Merge hwlocXYZ/hwloc/README-ompi.txt back into hwlocXYZ/README-ompi.txt instead of having both.
Add README.txt in new automake-required directory contrib/systemd/
Keep the following patches applied since they are not in 1.11.3
linux: actually enable libudev based on the result of AC_CHECK_LIB
(cherry picked from open-mpi/hwloc@9549fd59af)
configure: check the actual may_alias syntax that we use
(cherry picked from open-mpi/hwloc@0ab7af5e90)
This commit fixes several bugs in the registration cache code:
- Fix a programming error in the grdma invalidation function that can
cause an infinite loop if more than 100 registrations are
associated with a munmapped region. This happens because the
mca_rcache_base_vma_find_all function returns the same 100
registrations on each call. This has been fixed by adding an
iterate function to the vma tree interface.
- Always obtain the vma lock when needed. This is required because
there may be other threads in the system even if
opal_using_threads() is false. Additionally, since it is safe to do
so (the vma lock is recursive) the vma interface has been made
thread safe.
- Avoid calling free() while holding a lock. This avoids race
conditions with locks held outside the Open MPI code.
Fixesopen-mpi/ompi#1654.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Update external as well
Revise the change: we still need the MPI_Barrier in MPI_Finalize when we use a blocking fence, but do use the "lazy" wait for completion. Replace the direct logic in MPI_Init with a cleaner macro
WRAPPER_EXTRA flags are checked *before* the POST_CONFIG macro is
invoked. So set them in the main CONFIG macro.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit adds an additional check for 64-bit atomic support for __sync
builtins. If 64-bit support is not available the opal_atomic_*_64 atomics
are disabled.
Signed-off-by: Nathan Hjelm <hjelmn@me.com>
xlc 13.1.0 crashes because of our may_alias attributes in nolibxml.c
on Power7. libxml.c and nolibxml.c are the only may_alias users for now,
so change our configure check to match the actual code using it.
Thanks to Paul Hargrove for reporting and debugging the issue,
and providing the patch.
https://www.open-mpi.org/community/lists/devel/2016/05/18918.php
(cherry picked from open-mpi/hwloc@0ab7af5e90)
