Updated the OFI MTL's Recv cancel to be a non-blocking call to match
the MPI spec. Given fi_cancel succeeded, then it is expected that the
user will wait on the request to read the result of if the cancel has
completed.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com
(cherry picked from commit 25bdd118acd0f9d9c43662def0a00afde708992f)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
For the non thread-grouping paths, only the first (0th) OFI context
should be used for communication. Otherwise this would access a non existant
array item and cause segfault.
While at it, clarifiy some content regarding SEPs in README (Credit to Matias Cabral
for README edits).
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 6edcc479c4ed510bb14b6d47fc36cbe39c4a60e1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Replace all tabs with spaces. No code or logic changes.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit b556cabfe937f84f30e8870e0a8c128b839e5dfd)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Replace all tabs with spaces. No code or logic changes.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit aba2571881951a5fc88574125bc374a30f4cdd98)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
It doesn't seem like the BTL was using uninitialized pointer. But simply
setting the rcache pointer to NULL after destroying it makes the valgrind
errors go away.
Fixes Issue #6345
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 786e686d4347655b574e609c65626c8323bb49b2)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
When we exceed the threshold number of contexts created, print appropriate help
text
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 9cabcfdbba49f8b97f830f90e3d88be2f7685de4)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Provide the av_attr.count hint (number of addresses that will be
inserted into the address vector through the life of the process)
at initialization of the address vector. It's ok to be a bit
wrong, but some endpoints (RxR) can benefit by not going through
the slow growth realloc churn.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 44be7f139ac8e3130ffeb0afd4f43abdde31dd83)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With MTLs, there's no "other transport" when the remote side
does not have an active NIC, so we should print a useful error
message when the modex failed (indicating lack of a NIC on
the remote side).
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit fe25097194145234c175ee0487b1fa9b13658d49)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Moving to a model where we have users actively _enable_ SEP feature for use
rather than opening SEP by default if provider supports it. This allows us to
not regress (either functionally or for performance reasons) any apps that were
working correctly on regular endpoints.
Also, providing MCA to specify number of OFI contexts to create and default
this value to 1 (Given btl/ofi also creates one by default, this reduces the
incidence of a scenario where we allocate all available contexts by default and
if btl/ofi asks for one more, then provider breaks as it doesn't support it).
While at it, spruce up README on SEP content.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 37f9aff2a02e9d51628cc7075853ec4dba93d1e7)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
-> Added new targets in Makefile.am to call a new build script
generate-opt-funcs.pl to generate specialized functions for
each *.pm file.
-> Added new perl module *.pm files for send,isend,irecv,iprobe,improbe
which are loaded by generate-opt-funcs.pl to create new source files
that correspond to the name of the .pm file to be used as part of
MTL OFI.
-> Added mtl_ofi_opt.pm.template and updated README with details on the
specialization features and how to add additional specialization
support.
-> Added new opt_common/mtl_ofi_opt_common.pm containing common
functions for generating the specialized functions used by
all other *.pm modules.
-> Added new mtl_ofi.h which includes the definitions for the
function symbol table for storing the specialized functions along
with the definitions for the initialization functions for the
corresponding function pointers.
-> Based off the OFI provider capabilities the specialized function
pointers are assigned at mtl_ofi_component_init to the corresponding
MTL OFI function.
-> mca_mtl_ofi_module_t has been updated with the symbol table
struct which is assigned at component init.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
(cherry picked from commit bef5f50a42f53f4c6c30610aba236861f652f30e)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
For cases when the number of local processes is greater than the number of
available contexts, the SEP initialization phase would calculate the number of
contexts to provision for each rank to be 0 and would eventually crash.
Fix the issue here by using regular endpoints in the event the number of local
processes is more than available contexts. This fixes issue #6182.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit e5e19dfcf7618185bdc89f1e91506a9bf15355b1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Commit 109d0569ffd introduced a crash when an error occurred
before ofi_ctxt was allocated, including when no providers
passed the selection logic. Properly check that the pointer
is not NULL in the error cleanup code before dereferencing
the pointer.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 6e15128d960aaa40dd7e905ae3e9e53d9cdaac2a)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
OFI MTL supports OFI Scalable Endpoints feature as means to improve
multi-threaded application throughput and message rate. Currently the feature
is designed to utilize multiple TX/RX contexts exposed by the OFI provider in
conjunction with a multi-communicator MPI application model. For more
information, refer to README under mtl/ofi.
Reviewed-by: Matias Cabral <matias.a.cabral@intel.com>
Reviewed-by: Neil Spruit <neil.r.spruit@intel.com>
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 109d0569ffdc29f40518d02ad7a4d5bca3adc3d1)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
When an application is not using multiple threads to call into MPI, we can
safely ask for FI_THREAD_DOMAIN setting from the provider as it should
translate to the least amount of locking in provider.
Conversely, for applications using THREAD_MULTIPLE, explicitly ask for
FI_THREAD_SAFE to prevent race conditions.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit 5cbcae79d8619b99e4768578c8d11d3149e5c7c8)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
The 2 sided communication support is added for non-tagmatching provider
to take advantage of this BTL and PML OB1. The current state is
"functional" and not optimized for performance.
Two sided support is disabled by default and can be turned on by mca
parameter: "mca_btl_ofi_mode".
Signed-off-by: Thananon Patinyasakdikul <thananon.patinyasakdikul@intel.com>
(cherry picked from commit 080115d44069e0c461a1af105cd41f28849cdffc)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
OFI providers may reserve some of the upper bits of the tag for
internal usage and expose it using mem_tag_format. Check for that
and adjust communicator bits as needed.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
(cherry picked from commit d996f529c0377d794dea261c801e504dfbb33170)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Keep all comments in the user-facing mpi.h.in as "old style" C
comments: /* */. This gives us maximum portability, just on the off
chance that a user's C compiler does not support //-style comments.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit d522c270373264aff0a7a2066bc3163b09e9a94b)
1. __STDC_VERSION__ isn't necessarily defined (e.g., by C++
compilers). So check to make sure it is defined before we actually
check the value.
2. If we're in C++11 (or later), use static_assert().
3. Split the static assert macro in two macros:
* THIS_SYMBOL_WAS_REMOVED_IN_MPI30(...): Insert a valid expression
(i.e., 0, because it's only used with MPI_Datatype values, and
since MPI_Datatype is a pointer, 0 is a valid RHS expression)
before invoking the static assert so that we don't get a syntax
error instead of the actual static assert error.
* THIS_FUNCTION_WAS_REMOVED_IN_MPI30(...): No need for the valid
expression; just invoke the assert functionality.
Also remove an errant "\".
Thanks to Constantine Khrulev and Martin Audet for identifying the
issue and suggesting to use C11's static_assert().
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit 835f8f1834b8798a23ee0db6ad94315e30cb9be3)
We have been bad about updating the NEWS file in master with all
the changes that have gone into the release branches. Patch up
NEWS with the changes from v3.0, v3.1, and v4.0 branches.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 50765ae5a26f718c8840e65cb0cb813f4a65004b)
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
The NEWS file had a mix of ISO-8859-1 and UTF-8 encodings, which
was making a mess of decoding the non-ASCII characters in the
file. This patch unifies the NEWS file as a UTF-8 encoded file
and changes many of the places where we had ASCII-ified a persons
name.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
(cherry picked from commit 2e23893f04ef30c598a23889a4242f8cf4a45238)
Cherry-pick was modified to fix one more ISO-8859-1 character that
was in the v4.0.x branch but not in the master branch.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
These op codes used to be in bits/ipc.h but were removed in glibc in 2015
with a comment saying they should be defined in internal headers:
https://sourceware.org/bugzilla/show_bug.cgi?id=18560
and when glibc uses that syscall it seems to do so from its own definitions:
https://github.com/bminor/glibc/search?q=IPCOP_shmat&unscoped_q=IPCOP_shmat
So I think using #ifndef and defining them if they're not already defined
using the values from glibc is the best option.
At IBM it was the testing on redhat 8 that found this as an issue
(the opcodes being undefined on the system made the #define HAS_SHMDT
evaluate to false so intercept_shmat / intercept_shmdt were
left undefined so shmat/shmdt memory events went unintercepted).
(cherry picked from commit e8fab058dac7300569cb54b08e5500115f8bab8f)
Signed-off-by: Mark Allen <markalle@us.ibm.com>
correctly use strlen(char *) instead of sizeof(char *)
Thanks Georg Geiser for reporting this issue.
Refs. open-mpi/ompi#7772
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
(cherry picked from commit c450b2140540a1f8eae1a6e6f9a22d17cd40e7d8)
* Want to make sure that the result from `wc` is trimmed of spaces,
so the `0` check returns properly
* Add a few more comments, and fix wording in the warning message.
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
* `libevent_core.so` contains the core functionality that we depend upon
- `libevent.so` library has been identified as the legacy target.
- `libevent_core.so` exists as far back as Libevent 2.0.5 (oldest supported by OMPI)
* `libevent_pthreads.so` can work with either `-levent` or `-levent_core`
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
