Implements recursive doubling algorithm for MPI_Iallgather.
The algorithm can be used only for power-of-two number of processes.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
(cherry picked from commit a7386c1e09fb274991ca5b50d9d418a0d6b77b6c)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
An implementation of R. Rabenseifner's algorithm for MPI_Ireduce.
This algorithm is a combination of a reduce-scatter implemented with recursive vector halving
and recursive distance doubling, followed either by a gather.
Limitations:
-- count >= 2^{\floor{\log_2 p}}
-- commutative operations only
-- intra-communicators only
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
(cherry picked from commit 7bd63e79c865080c801a45ed852602bdc4eb4d8f)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Remove dead code that was causing warnings about unused static
functions.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 2e24e6ec082d29f76dcbc75c6f214d2d0d647701)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Implements recursive doubling algorithm for MPI_Iexscan.
The algorithm preserves order of operations so it can be used both
by commutative and non-commutative operations.
The MCA parameter 'coll_libnbc_iexscan_algorithm' was added for dynamic
algorithm selection.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
(cherry picked from commit dfe203e167f5d8abc3b55226c6f17a468c9567dd)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Implements recursive doubling algorithm for MPI_Iscan. The algorithm preserves order of operations so it can be used both by commutative and non-commutative operations.
The MCA parameter coll_libnbc_iscan_algorithm was added for dynamic algorithm selection.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
(cherry picked from commit 3d43ff0f3209d5bf4713c6696acb0acb8f1756e4)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Gcc 8 identified hb_tree_csearch() as an infinite recursion, and it
turns out that we never call this function, anyway. So just remove
it.
Fixes#5670.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit 06c1bf73da875f4a6449f38a993530d6fae7817d)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
always initialize 'size'.
Only the a2a_sched_diss() alltoall algorithm is impacted,
and this algo is currently unused, so there is no need
to backport nor update the NEWS file for now.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
(cherry picked from commit ff48e9286430b37aac3146efe2b355f255db94d5)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
The call of MPI_Allgatherv with sendbuf and sendtype parameters equal to MPI_IN_PLACE and NULL correspondingly, produces the segmentation fault.
The problem is that sendtype is used even when sendbuf value is MPI_IN_PLACE. But according to the standard, sendtype and sendcount parameters should be ignored in this case.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
(cherry picked from commit b45e190e6629b664872f7f872cfffd916180bb9a)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
also add common verbose variable.
Note the verbosity thing is a little tricky owing to the way the MCA frameworks and components are registered and
and initialized. The BTL's are registered/initialized prior to the MTL components even getting registered.
Here's the change in ofi mtl mca parameters. Before commit:
MCA mtl ofi: parameter "mtl_ofi_provider_include" (current value: "psm2", data source: environment, level: 1 user/basic, type: string)
Comma-delimited list of OFI providers that are considered for use (e.g., "psm,psm2"; an empty value means that all providers will be considered). Mutually exclusive with mtl_ofi_provider_exclude.
MCA mtl ofi: parameter "mtl_ofi_provider_exclude" (current value: "shm,sockets,tcp,udp,rstream", data source: default, level: 1 user/basic, type: string)
Comma-delimited list of OFI providers that are not considered for use (default: "sockets,mxm"; empty value means that all providers will be considered). Mutually exclusive with mtl_ofi_provider_include.
After commit:
MCA btl ofi: parameter "btl_ofi_provider_include" (current value: "", data source: default, level: 1 user/basic, type: string, synonym of: opal_common_ofi_provider_include)
Comma-delimited list of OFI providers that are considered for use (e.g., "psm,psm2"; an empty value means that all providers will be considered). Mutually exclusive with mtl_ofi_provider_exclude.
MCA btl ofi: parameter "btl_ofi_provider_exclude" (current value: "shm,sockets,tcp,udp,rstream", data source: default, level: 1 user/basic, type: string, synonym of: opal_common_ofi_provider_exclude)
Comma-delimited list of OFI providers that are not considered for use (default: "sockets,mxm"; empty value means that all providers will be considered). Mutually exclusive with mtl_ofi_provider_include.
MCA mtl ofi: parameter "mtl_ofi_provider_exclude" (current value: "shm,sockets,tcp,udp,rstream", data source: default, level: 1 user/basic, type: string, synonym of: opal_common_ofi_provider_exclude)
Comma-delimited list of OFI providers that are not considered for use (default: "sockets,mxm"; empty value means that all providers will be considered). Mutually exclusive with mtl_ofi_provider_include.
MCA mtl ofi: parameter "mtl_ofi_verbose" (current value: "0", data source: default, level: 3 user/all, type: int, synonym of: opal_common_ofi_verbose)
related to #7755
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
(cherry picked from commit 9f1081a07ac3c7b7277a27277ed970ed713207c9)
(cherry picked from commit 45b643d0cfa46f1abb9a5f43cf0ff304cf6a5fea)
This commit fixes an issue with non-debug builds where adding an
attachment to the attachment list doesn't actually happen. This
causes all MPI_Win_detach calls to fail. The call was within an
assert which is optimized out in optimized builds.
Signed-off-by: Nathan Hjelm <hjelmn@google.com>
(cherry picked from commit 8ee80d885561c4349ab97725416613614d63c77f)
Adds the capability to select a NIC based on hardware locality.
Creates a list of NICs that share the same cpuset as the process,
then selects the NIC based on the (local rank) % (number of NICs).
If no NICs are available that share the same cpuset, the selection process
will create a list of all available NICs and make a selection based on
(local rank) % (number of NICs)
Signed-off-by: Nikola Dancejic <dancejic@amazon.com>
(cherry picked from commit 167d75b42ac3ca4770d59c796c011d72e0fffde3)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Change ompi_mtl_ofi_get_endpoint() to call the active PML's
add_procs() rather than the OFI MTL add_procs() directly when
discovering a new process during operation.
Functionally, this has no impact in correct operation. However,
the current behavior means that the heterogenous and active PML
checks are not being executed in the dynamic discovery case.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 64d70b3076186b044f39f11d0d7d3e5bbcaec469)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Some versions of Libfabric contain a bug in EFA where FI_REMOTE_COMM and
FI_LOCAL_COMM are not advertised. In order to workaround this, we need to call
fi_getinfo() without those capability bits to see if EFA is available first.
Also move around some of the provider include/exclude list logic so we can skip
this workaround if applicable.
Signed-off-by: Robert Wespetal <wesper@amazon.com>
(cherry picked from commit 49128a7adbf197027043994f75cd01acb71f2a98)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Make sure to get an RDM provider that can provide both local and
remote communication. We need this check because some providers could
be selected via RXD or RXM, but can't provide local communication, for
example.
Add OPAL_CHECK_OFI_VERSION_GE() m4 macro to check that the Libfabric
we're building against is >= a target version. Use this check in two
places:
1. MTL/OFI: Make sure it is >= v1.5, because the FI_LOCAL_COMM /
FI_REMOTE_COMM constants were introduced in Libfabric API v1.5.
2. BTL/usnic: It already had similar configury to check for Libfabric
>= v1.1, but the usnic component was checking for >= v1.3. So
update the btl/usnic configury to use the new macro and check for
>= v1.3.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit 21bc9042e1ff9c3075ababda9eaf49ccc90f64db)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Updated the OFI MTL's Recv cancel to be a non-blocking call to match
the MPI spec. Given fi_cancel succeeded, then it is expected that the
user will wait on the request to read the result of if the cancel has
completed.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com
(cherry picked from commit 25bdd118acd0f9d9c43662def0a00afde708992f)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
For the non thread-grouping paths, only the first (0th) OFI context
should be used for communication. Otherwise this would access a non existant
array item and cause segfault.
While at it, clarifiy some content regarding SEPs in README (Credit to Matias Cabral
for README edits).
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 6edcc479c4ed510bb14b6d47fc36cbe39c4a60e1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Replace all tabs with spaces. No code or logic changes.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit aba2571881951a5fc88574125bc374a30f4cdd98)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
When we exceed the threshold number of contexts created, print appropriate help
text
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 9cabcfdbba49f8b97f830f90e3d88be2f7685de4)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Provide the av_attr.count hint (number of addresses that will be
inserted into the address vector through the life of the process)
at initialization of the address vector. It's ok to be a bit
wrong, but some endpoints (RxR) can benefit by not going through
the slow growth realloc churn.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 44be7f139ac8e3130ffeb0afd4f43abdde31dd83)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With MTLs, there's no "other transport" when the remote side
does not have an active NIC, so we should print a useful error
message when the modex failed (indicating lack of a NIC on
the remote side).
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit fe25097194145234c175ee0487b1fa9b13658d49)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Moving to a model where we have users actively _enable_ SEP feature for use
rather than opening SEP by default if provider supports it. This allows us to
not regress (either functionally or for performance reasons) any apps that were
working correctly on regular endpoints.
Also, providing MCA to specify number of OFI contexts to create and default
this value to 1 (Given btl/ofi also creates one by default, this reduces the
incidence of a scenario where we allocate all available contexts by default and
if btl/ofi asks for one more, then provider breaks as it doesn't support it).
While at it, spruce up README on SEP content.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 37f9aff2a02e9d51628cc7075853ec4dba93d1e7)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
-> Added new targets in Makefile.am to call a new build script
generate-opt-funcs.pl to generate specialized functions for
each *.pm file.
-> Added new perl module *.pm files for send,isend,irecv,iprobe,improbe
which are loaded by generate-opt-funcs.pl to create new source files
that correspond to the name of the .pm file to be used as part of
MTL OFI.
-> Added mtl_ofi_opt.pm.template and updated README with details on the
specialization features and how to add additional specialization
support.
-> Added new opt_common/mtl_ofi_opt_common.pm containing common
functions for generating the specialized functions used by
all other *.pm modules.
-> Added new mtl_ofi.h which includes the definitions for the
function symbol table for storing the specialized functions along
with the definitions for the initialization functions for the
corresponding function pointers.
-> Based off the OFI provider capabilities the specialized function
pointers are assigned at mtl_ofi_component_init to the corresponding
MTL OFI function.
-> mca_mtl_ofi_module_t has been updated with the symbol table
struct which is assigned at component init.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
(cherry picked from commit bef5f50a42f53f4c6c30610aba236861f652f30e)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
For cases when the number of local processes is greater than the number of
available contexts, the SEP initialization phase would calculate the number of
contexts to provision for each rank to be 0 and would eventually crash.
Fix the issue here by using regular endpoints in the event the number of local
processes is more than available contexts. This fixes issue #6182.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit e5e19dfcf7618185bdc89f1e91506a9bf15355b1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Commit 109d0569ffd introduced a crash when an error occurred
before ofi_ctxt was allocated, including when no providers
passed the selection logic. Properly check that the pointer
is not NULL in the error cleanup code before dereferencing
the pointer.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 6e15128d960aaa40dd7e905ae3e9e53d9cdaac2a)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
OFI MTL supports OFI Scalable Endpoints feature as means to improve
multi-threaded application throughput and message rate. Currently the feature
is designed to utilize multiple TX/RX contexts exposed by the OFI provider in
conjunction with a multi-communicator MPI application model. For more
information, refer to README under mtl/ofi.
Reviewed-by: Matias Cabral <matias.a.cabral@intel.com>
Reviewed-by: Neil Spruit <neil.r.spruit@intel.com>
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 109d0569ffdc29f40518d02ad7a4d5bca3adc3d1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
When an application is not using multiple threads to call into MPI, we can
safely ask for FI_THREAD_DOMAIN setting from the provider as it should
translate to the least amount of locking in provider.
Conversely, for applications using THREAD_MULTIPLE, explicitly ask for
FI_THREAD_SAFE to prevent race conditions.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 5cbcae79d8619b99e4768578c8d11d3149e5c7c8)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
OFI providers may reserve some of the upper bits of the tag for
internal usage and expose it using mem_tag_format. Check for that
and adjust communicator bits as needed.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit d996f529c0377d794dea261c801e504dfbb33170)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
As discussed, a feature is being added to libpsm2 to correctly handle
the case where the library is opened by multiple OMPI transports in the same
process. (For example, the OFI BTL and the PSM2 MTL).
* Improved error message to indicate required libpsm2 version.
* Adds a test at autogen/configure time for the existence of
PSM2_LIB_REFCOUNT_CAP.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Signed-off-by: Michael Heinz <michael.william.heinz@intel.com>
(cherry picked from commit f10305a49facec8daef24ac81a52207a3f4fb73f)
This is a fix based on a bugreport on github/mailing list from CGNS.
The core of the problem was that different processes entered different branches of
our aggregator selection logic, due to the fact that in some cases processes had
a matching file_view size and contiguous chunk size (thus assuming 1-D distribution),
and some processes did not (thus assuming 2-D distribution). The fix is to calculate
the avg. file view size across all processes and use this value, thus ensuring that
all processes enter the same branch.
Fixes issue #7809
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
(cherry picked from commit 4a8a330bbaf9fe5ea07cd01146afb83b569f3138)
Keep all comments in the user-facing mpi.h.in as "old style" C
comments: /* */. This gives us maximum portability, just on the off
chance that a user's C compiler does not support //-style comments.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit d522c270373264aff0a7a2066bc3163b09e9a94b)
1. __STDC_VERSION__ isn't necessarily defined (e.g., by C++
compilers). So check to make sure it is defined before we actually
check the value.
2. If we're in C++11 (or later), use static_assert().
3. Split the static assert macro in two macros:
* THIS_SYMBOL_WAS_REMOVED_IN_MPI30(...): Insert a valid expression
(i.e., 0, because it's only used with MPI_Datatype values, and
since MPI_Datatype is a pointer, 0 is a valid RHS expression)
before invoking the static assert so that we don't get a syntax
error instead of the actual static assert error.
* THIS_FUNCTION_WAS_REMOVED_IN_MPI30(...): No need for the valid
expression; just invoke the assert functionality.
Also remove an errant "\".
Thanks to Constantine Khrulev and Martin Audet for identifying the
issue and suggesting to use C11's static_assert().
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit 835f8f1834b8798a23ee0db6ad94315e30cb9be3)
- added detection of new API into configuration
- added tag_send call implemented using new API
- added MPI_Send/MPI_Isend/MPI_Recv/MPI_Irecv implementations
Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
(cherry picked from commit 75bda25ddbeea18bb001f367a712dc72592e1e58)
The locality for remote procs is not provided as it is only a local
concept. Thus, you must always use modex_recv_optional to ensure you
don't hang waiting for a response until dmodex times out.
Signed-off-by: Ralph Castain <rhc@pmix.org>
An incorrect backport in open-mpi/ompi#7360 removed
constants.c from ompi/mpi/fortran/use-mpi-f08/base/Makefile.am
This one off commit fixes that, and move constants.h from
ompi/mpi/fortran/use-mpi-f08 to ompi/mpi/fortran/use-mpi-f08/base
Fixesopen-mpi/ompi#7616
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
