- there was a set of UCX related issues reported which caused
by mmap API hooks conflicts. We added diagnostic of such
problems to simplify bug-resolving pipeline
Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
in schizo/ompi, sets the new OMPI_MCA_mpi_oversubscribe environment
variable according to the node oversubscription state.
This MCA parameter is used to set the default value of the
mpi_yield_when_idle parameter.
This two steps tango is needed so the mpi_yield_when_idle setting
is always honored when set in a config file.
Refs. open-mpi/ompi#6433
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
This commit DELETES the removed MPI1 functions and datatypes from
both the mpi.h header and from the library (they were deleted from the
MPI standard in MPI-3.0).
WARNING: This changes the MPI API in a non-backwards compatible way.
This also removes the configure option that was added in Open
MPI v4.0.x, requiring users to change their apps if they are
using any of these almost 20 year old APIs.
This commit removes the following MPI1 removed functions and datatypes:
MPI_Address
MPI_Errhandler_create
MPI_Errhandler_get
MPI_Errhandler_set
MPI_Type_extent
MPI_Type_hindexed
MPI_Type_hvector
MPI_Type_struct
MPI_Type_UB
MPI_Type_LB
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
Refs https://github.com/open-mpi/ompi/issues/6278.
This commit is intended to be cherry-picked to v4.0.x and
the following commit will ammend to this functionality for
master's removal.
Changes the prototypes for MPI removed functions in the
following ways:
There are 4 cases:
1) User wants MPI-1 compatibility (--enable-mpi1-compatibility)
MPI_Address (and friends) are declared in mpi.h with
deprecation notice
2) User does not want MPI-1 compatibility, and has a C11-capable
compiler
Declare an MPI_Address (etc.) macro in mpi.h, which will
cause a compile-time error using _Static_assert C11 feature
3) User does not want MPI-1 compatibility, and does not have a
C11-capable compiler, but the compiler supports error function
attributes.
Declare an MPI_Address (etc.) macro in mpi.h, which will
cause a compile-time error using error function attribute.
4) User does not want MPI-1 compatibility, and does not have a
C11-capable compiler, or a compiler that supports error
function attributes.
Do not declare MPI_Address (etc.) in mpi.h at all.
Unless the user is compiling with something like -Werror,
this will allow the user's code to compile. We are
choosing this because it seems like a losing battle to
make some kind of compile time error that is friendly to
the user (and doesn't make it look like mpi.h itself is broken).
On v4.0.x, this will allow the user code to both compile
(albeit with a warning) and link (because the MPI_Address
will be in the MPI library because we are preserving ABI
back to 3.0.x).
On master/v5.0.x, this will allow the user code to compile,
but it will fail to link (because the MPI_Address symbol will
not be in the MPI library).
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
commit c6070fd2e broke building fortran bindings
with PGI compilers. Turns out PGI compilers need
to link in the *.o from a module file whether or
not there are module subroutines defined or not in
the module file.
Related to #6411
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
mark the "self" peer OMPI_OSC_RDMA_PEER_LOCAL_BASE when
the window is dynamically created and use_cpu_atomics is set
in order to correctly handle communications to self.
Thanks Bart Janssens for reporting this issue.
Refs. open-mpi/ompi#6394
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Place the content of common_ucx_int.h back to the common_ucx.h and
include common_ucx_wpool.h explicitly.
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
Updated the OFI MTL's Recv cancel to be a non-blocking call to match
the MPI spec. Given fi_cancel succeeded, then it is expected that the
user will wait on the request to read the result of if the cancel has
completed.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com
For remote node peers pack smaller worker address, which contains
network device addresses only. This would reduce amount of OOB traffic
during startup.
Signed-off-by: Mikhail Brinskii <mikhailb@mellanox.com>
We've been fighting the battle of trying to create a regex generator and
parser that can handle arbitrary hostname schemes - without long-term
success. The worst of it is that there is no way of checking to see if
the computed regex is correct short of parsing it and doing a
character-by-character comparison with the original string. Ugh...there
has to be a better solution.
One option is to investigate using 3rd-party regex libraries as
those are coming from communities whose sole focus is resolving that
problem. However, someone would need to spend the time to investigate
it, and we'd have to find a license-friendly implementation.
Another option is to quit beating our heads against the wall and just
compress the information. It won't be as much of a reduction, but we
also won't keep hitting scenarios where things break. In this case, it
seems that "perfection" is definitely the enemy of "good enough".
This PR implements the compression option while retaining the
possibility of people adding regex-generating components. The
compression code used in ORTE is consolidated into the opal/compress
framework. That framework currently held bzip and gzip components for
use in compressing checkpoint files - since we no longer support C/R, I
have .opal_ignore'd those components.
However, I have left the original framework APIs alone in case someone
ever decides to redo C/R. The APIs of interest here are added to the
framework - specifically, the "compress_block" and "decompress_block"
functions. I then moved the ORTE zlib compression code into a new
component in this framework.
Unfortunately, the framework currently is a single-select one - i.e.,
only one active component at a time. Since I .opal_ignore'd the other
two and made the priority of zlib high, this isn't a problem. However,
if someone wants to re-enable bzip/gzip or add another component, they
might need to transition opal/compress to a multi-select framework.
Included changes:
* Consolidate the compression code into the opal/compress framework
* Move the ORTE zlib compression code into a new opal/compress/zlib
component
* Ignore the bzip and gzip components in opal/compress framework
* Add a "compress_base_limit" MCA param to set the threshold above which
we compress data - defaults to 4096 bytes
* Delete stale brucks and rcd components from orte/grpcomm framework
* Delete the orte/regx framework
* Update the launch system to use opal/compress instead of string regex
* Provide a default module if no zlib is available
* Fix some misc multi-node issues
* Properly generate the nidmap in response to a "connection warmup"
message so the remote daemon knows the children it needs to launch.
* Remove stale references to orte_node_regex
* opal_byte_object_t's are not OPAL objects - properly release allocated
memory.
* Set the topology
* Currently only handling homogeneous case
* Update the compress framework files to conform
* Consolidate open/close into one "frame" file. Ensure we open/close the
framework
Signed-off-by: Ralph Castain <rhc@pmix.org>
For the non thread-grouping paths, only the first (0th) OFI context
should be used for communication. Otherwise this would access a non existant
array item and cause segfault.
While at it, clarifiy some content regarding SEPs in README (Credit to Matias Cabral
for README edits).
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
Update the OPAL_CHECK_OFI configury macro:
- Make it safe to call the macro multiple times:
- The checks only execute the first time it is invoked
- Subsequent invocations, it just emits a friendly "checking..."
message so that configure output is sensible/logical
- With the goal of ultimately removing opal/mca/common/ofi, rename the
output variables from OPAL_CHECK_OFI to be
opal_ofi_{happy|CPPFLAGS|LDFLAGS|LIBS}.
- Update btl/ofi, btl/usnic, and mtl/ofi for these new conventions.
- Also, don't use AC_REQUIRE to invoke OPAL_CHECK_OFI because that
causes the macro to be invoked at a fairly random time, which makes
configure stdout confusing / hard to grok.
- Remove a little left-over kruft in OPAL_CHECK_OFI, too (which
resulted in an indenting change, making the change to
opal_check_ofi.m4 look larger than it really is).
Thanks Alastair McKinstry for the report and initial fix.
Thanks Rashika Kheria for the reminder.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
According to the MPI standard the obj_handle is a pointer to an MPI
object, and therefore cannot be MPI_COMM_WORLD. The MPI standard example
14.6 highlight this usage.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
`MPIX_C_FLOAT16` is defined as a synonym for `MPIX_SHORT_FLOAT`
if the C compiler supports `_Float16`, which is defined in
ISO/IEC JTC 1/SC 22/WG 14 N1945 (ISO/IEC TS 18661-3:2015).
This name and meaning are same as that of MPICH. This may be
a transitional datatype until the MPI Forum decides a proper
name for the type.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
This extension provides additional MPI datatypes `MPIX_SHORT_FLOAT`,
`MPIX_C_SHORT_FLOAT_COMPLEX`, and `MPIX_CXX_SHORT_FLOAT_COMPLEX`
for `short float` (C/C++), `short float _Complex` (C), and
`std::complex<short float>` (C++), respectively, or their alternate
types like `_Float16`.
See `ompi/mpiext/shortfloat/README.txt` for details.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
... and add `MPI_COMPLEX4`.
This commit changes values of existing `OMPI_DATATYPE_MPI_*` macros.
This change does not affect ABI compatibility of `libmpi.so` and the
like because these values are only used in OMPI internal code.
On the other hand, `ompi_datatype_t::id` values of existing datatypes
are not changed and 73 is newly assigned to for `MPI_COMPLEX4` to
retain ABI compatibility.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
... and `ompi_mpi_c_short_float_complex` and `ompi_mpi_cxx_sfltcplex`.
These are Open MPI internal variables intended to be defined as
`MPI_SHORT_FLOAT`, `MPI_C_SHORT_FLOAT_COMPLEX`, and
`MPI_CXX_SHORT_FLOAT_COMPLEX` in the future.
`OMPI_DATATYPE_MPI_C_SHORT_FLOAT_COMPLEX` is also required to
support `MPI_COMPLEX4` in the next commit.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The type `short float`, which is proposed in ISO/IEC JTC 1/SC 22 WG 14
(C WG), is not supported by most compilers yet. But some compilers
(including gcc 7 for AArch64 and clang 6) support `_Float16`, which
is defined in ISO/IEC TS 18661-3:2015 (ISO/IEC JTC 1/SC 22/WG 14 N1945)
as an extensions for C. If it is detected in `configure`, it is used
as an alternate type of `short float` in Open MPI internal code.
This commit adds a `configure` option `--enable-alt-short-float=TYPE`.
It can be used to specify a type other than `short float` and `_Float16`
as the alternate type.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The type `short float` is proposed for the C language in ISO/IEC JTC
1/SC 22 WG 14 (C WG) for mainly IEEE 754-2008 binary16, a.k.a.
half-precision floating point or FP16.
By this commit, `short float` and `short float _Complex` are detected
in `configure` and used in Open MPI internal code. `MPI_SHORT_FLOAT`
and its complex number version are not added yet.
This commit changes values of existing `OPAL_DATATYPE_*` macros.
This change does not affect ABI compatibility of `libmpi.so` and the
like because these values are only used in OPAL and OMPI internal code.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
When compiling mpi.h with a modern C++ compiler and a high degree of
pickyness (e.g., -Wold-style-cast), casting using (void*) in the
OMPI_PREDEFINED_GLOBAL and MPI_STATUS*_IGNORE macros will emit
warnings. So if we're compiling with a C++ compiler, use C++'s
static_cast<> instead of (void*).
Thanks to @shadow-fax for identifying the issue.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
treematch/km_partitioning.c #include "config.h",
but there is no such file when the embedded treematch is used.
In order to prevent the embedded treematch from incorrectly using
the config.h from the embedded hwloc, generate a dummy config.h.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
When we exceed the threshold number of contexts created, print appropriate help
text
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
We missed an assert to check if ALLOW_OVERTAKE is set or not before
validating the sequence number and this will cause deadlock.
Signed-off-by: Thananon Patinyasakdikul <tpatinya@utk.edu>
ACCUMULATE, unlike REDUCE, can use with derived
datatypes with predefinied operations, with some
restrictions outlined in MPI-3:11.3.4. The derived
datatype must be composed entierly from one predefined
datatype (so you can do all the construction you want,
but at the bottom, you can only use one datatype, say,
MPI_INT).
Refs. open-mpi/ompi#6275
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Provide the av_attr.count hint (number of addresses that will be
inserted into the address vector through the life of the process)
at initialization of the address vector. It's ok to be a bit
wrong, but some endpoints (RxR) can benefit by not going through
the slow growth realloc churn.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
This commit fixes a problem reported on the mailing list with
individual writes larger than 512 MB.
The culprit is a floating point division of two large, close values.
Changing the datatypes from float to double (which is what is being
used in the fcoll components) fixes the problem.
See issue #6285 and
https://forum.hdfgroup.org/t/cannot-write-more-than-512-mb-in-1d/5118
Thanks for Axel Huebl and René Widera for reporting the issue.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
With MTLs, there's no "other transport" when the remote side
does not have an active NIC, so we should print a useful error
message when the modex failed (indicating lack of a NIC on
the remote side).
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Similar to #6286 rounding number of bytes into a single precision floating point value to round up the result of a division is a potential risk due to rounding errors.
- remove floating point operations for `round up`
- removes floating point conversion for round down (native behavior of integer division)
Signed-off-by: René Widera <r.widera@hzdr.de>
If MPI extensions are enabled, all
`ompi/mpiext/pcollreq/use-mpi/mpiext_*_usempi.h` are included in
`ompi/mpi/fortran/mpiext-use-mpi/mpi-ext-module.F90` and all
`ompi/mpiext/pcollreq/use-mpi/mpiext_*_usempif08.h` are included in
`ompi/mpi/fortran/mpiext-use-mpi-f08/mpi-f08-ext-module.F90` using
`#include` directives.
In `mpiext_*_usempi.h` and `mpiext_*_usempif08.h`, some MPI extension
may want to use constants or handles defined in the `mpi` module and
the `mpi_f08` module. For example, if you want to define a new
datatype in `mpi_f08_ext`, you'll need the definition of
`type(mpi_datatype)`. However, putting `use mpi_f08` line in thier
`mpiext_*_usempif08.h` may cause a compilation error if more than
one MPI extensions are enabled because the `use` statement must be
put prior to any variable declarations.
To resolve this problem, this commit puts `use mpi` and `use mpi_f08`
as first lines of `mpi-ext-module.F90` and `mpi-f08-ext-module.F90`
respectively.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Including `mpiext_*_mpifh.h` in the source file of the `mpi_f08_ext`
module is not always appropriate. For example, if you want to define
a new datatype in an MPI extension, the `include 'mpif-ext.h'` binding
defines the datatype as `integer` but the `use mpi_f08_ext` binding
defines it as `type(mpi_datatype)`. They conflict.
This commit allows each MPI extension to declare whether it wants to
include its `mpiext_*_mpifh.h` in `mpi_f08` and `mpi_f08_ext`
respectively. The default (no declaration) is 'want'.
See `ompi/mpiext/example/configure.m4` for an example.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Moving to a model where we have users actively _enable_ SEP feature for use
rather than opening SEP by default if provider supports it. This allows us to
not regress (either functionally or for performance reasons) any apps that were
working correctly on regular endpoints.
Also, providing MCA to specify number of OFI contexts to create and default
this value to 1 (Given btl/ofi also creates one by default, this reduces the
incidence of a scenario where we allocate all available contexts by default and
if btl/ofi asks for one more, then provider breaks as it doesn't support it).
While at it, spruce up README on SEP content.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
Valgrind warns that *newtype is uninitialized when calling from
Fortran as e.g.
use mpi
integer :: t, err
call MPI_Type_create_f90_integer(5, t, err)
Since newtype is intent(out), this should not happen. There is
no reason to convert the type using PMPI_Type_f2c, only to over-
write it immediately afterwards. The other type_create_* functions
did not convert newtype.
The valgrind warnings:
==28441== Conditional jump or move depends on uninitialised value(s)
==28441== at 0x581B555: PMPI_Type_f2c (in [...]/lib/libmpi.so.0.0.0)
==28441== by 0x4E87AB7: MPI_TYPE_CREATE_F90_INTEGER (in [...]/lib/libmpi_mpifh.so.0.0.0)
==28441== by 0x400BA1: MAIN__ (in [...])
==28441== by 0x400C46: main (in [...])
==28441==
==28441== Conditional jump or move depends on uninitialised value(s)
==28441== at 0x581B563: PMPI_Type_f2c (in [...]/lib/libmpi.so.0.0.0)
==28441== by 0x4E87AB7: MPI_TYPE_CREATE_F90_INTEGER (in [...]/lib/libmpi_mpifh.so.0.0.0)
==28441== by 0x400BA1: MAIN__ (in [..])
==28441== by 0x400C46: main (in [...])
==28441==
==28441== Use of uninitialised value of size 8
==28441== at 0x581B577: PMPI_Type_f2c (in [...]/lib/libmpi.so.0.0.0)
==28441== by 0x4E87AB7: MPI_TYPE_CREATE_F90_INTEGER (in [...]/lib/libmpi_mpifh.so.0.0.0)
==28441== by 0x400BA1: MAIN__ (in [...])
==28441== by 0x400C46: main (in [...])
==28441==
Signed-off-by: Risto Toijala <risto.toijala@gmail.com>
opal_string_copy() takes care of all the string computations.
Specifically: when we converted to opal_string_copy(), we accidentally
left the *source* length as the argument, not the *target* length,
which resulted in one less character being copied than intended (as
was showing up in MTT C++ testing results).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This is a holdover from LAM/MPI that was never implemented here in
Open MPI (and never will be). Might as well remove this dead code.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit fixes a bug when launching with prun where the process
info structures used by the btls are not populated.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
-> Added new targets in Makefile.am to call a new build script
generate-opt-funcs.pl to generate specialized functions for
each *.pm file.
-> Added new perl module *.pm files for send,isend,irecv,iprobe,improbe
which are loaded by generate-opt-funcs.pl to create new source files
that correspond to the name of the .pm file to be used as part of
MTL OFI.
-> Added mtl_ofi_opt.pm.template and updated README with details on the
specialization features and how to add additional specialization
support.
-> Added new opt_common/mtl_ofi_opt_common.pm containing common
functions for generating the specialized functions used by
all other *.pm modules.
-> Added new mtl_ofi.h which includes the definitions for the
function symbol table for storing the specialized functions along
with the definitions for the initialization functions for the
corresponding function pointers.
-> Based off the OFI provider capabilities the specialized function
pointers are assigned at mtl_ofi_component_init to the corresponding
MTL OFI function.
-> mca_mtl_ofi_module_t has been updated with the symbol table
struct which is assigned at component init.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
For cases when the number of local processes is greater than the number of
available contexts, the SEP initialization phase would calculate the number of
contexts to provision for each rank to be 0 and would eventually crash.
Fix the issue here by using regular endpoints in the event the number of local
processes is more than available contexts. This fixes issue #6182.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
Commit 109d0569ff introduced a crash when an error occurred
before ofi_ctxt was allocated, including when no providers
passed the selection logic. Properly check that the pointer
is not NULL in the error cleanup code before dereferencing
the pointer.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Though the MPI standard does not have `MPI_CXX_COMPLEX`, `mpi.h`,
`mpif.h`, and `mpi.mod` have it. So I added it for consistency.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
This commit fixes edge cases of `r = 38` and `r = 308`.
As defined in the MPI standard, `TYPE_CREATE_F90_REAL` and
`TYPE_CREATE_F90_COMPLEX` must be consistent with the Fortran
`SELECTED_REAL_KIND` function. The `SELECTED_REAL_KIND` function is
defined based on the `RANGE` function. The `RANGE` function returns
`INT(MIN(LOG10(HUGE(X)), -LOG10(TINY(X))))` for a real value `X`.
The old code considers only `INT(LOG10(HUGE(X)))` using `*_MAX_10_EXP`.
This commit adds `INT(-LOG10(TINY(X)))` part using `*_MIN_10_EXP`.
This bug affected the following `p`-`r` combinations.
| p | r | expected | returned | expected | returned |
| :------------ | --: | :-------- | :-------- | :------- | :-------- |
| MPI_UNDEFINED | 38 | REAL8 | REAL4 | COMPLEX16 | COMPLEX8 |
| 0 <= p <= 6 | 38 | REAL8 | REAL4 | COMPLEX16 | COMPLEX8 |
| MPI_UNDEFINED | 308 | REAL16 | REAL8 | COMPLEX32 | COMPLEX16 |
| 0 <= p <= 15 | 308 | REAL16 | REAL8 | COMPLEX32 | COMPLEX16 |
MPICH returns the same result as Open MPI with this fix.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Per discussion on https://github.com/open-mpi/ompi/pull/6030
and https://github.com/open-mpi/ompi/pull/6145, move
around where MPI extension header files are installed (specifically:
the installation tree path does not need to match the source tree
path).
For reference, header files were installed like this :
- <prefix>/include/openmpi/ompi/mpiext/pcollreq/mpif-h/mpiext_pcollreq_mpifh.h
- <prefix>/include/openmpi/ompi/mpiext/pcollreq/c/mpiext_pcollreq_c.h
and they are now installed like this :
- <prefix>/include/openmpi/mpiext/mpiext_pcollreq_mpifh.h
- <prefix>/include/openmpi/mpiext/mpiext_pcollreq_c.h
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
The intention of lowering the priority when all processes are local
was to favor Vader BTL. However, in builds including the OFI MTL it
gets selected instead.
Reviewed-by: Spruit, Neil R <neil.r.spruit@intel.com>
Reviewed-by: Gopalakrishnan, Aravind <aravind.gopalakrishnan@intel.com>
Signed-off-by: Matias Cabral <matias.a.cabral@intel.com>
OFI MTL supports OFI Scalable Endpoints feature as means to improve
multi-threaded application throughput and message rate. Currently the feature
is designed to utilize multiple TX/RX contexts exposed by the OFI provider in
conjunction with a multi-communicator MPI application model. For more
information, refer to README under mtl/ofi.
Reviewed-by: Matias Cabral <matias.a.cabral@intel.com>
Reviewed-by: Neil Spruit <neil.r.spruit@intel.com>
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
move openmpi/ompi/mpiext/FOO/c/mpiext_FOO_c.h to
openmpi/ompi/mpiext/FOO_c.h in order to use consistent
paths with mpif.h extensions
Refs. open-mpi/ompi#6019
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
in order to cope with the 72 characters per line limit, move
openmpi/ompi/mpiext/FOO/mpif-h/mpiext_FOO_mpifh.h to
openmpi/ompi/mpiext/FOO_mpifh.h
Refs. open-mpi/ompi#6019
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
The feature of persistent collectives is approved in the Sept. 2018
MPI Forum meeting and 2018 Draft Specification of the MPI standard is
published during SC18.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
without this fix, an error handler invoked on pml_ucx request would
segfault while trying to dereference requests[i]->req_mpi_object.comm
Signed-off-by: Yossi Itigin <yosefe@mellanox.com>
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
Correctly bubble up errors in NBC collective operations
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
The error field of requests needs to be rearmed at start, not at create
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
So far Vader is faster than OFI MTL for doing shared memory.
Therefore, let it run by default when all procs are local.
Reviewed-by: Spruit, Neil R <neil.r.spruit@intel.com>
Reviewed-by: Gopalakrishnan, Aravind <aravind.gopalakrishnan@intel.com>
Signed-off-by: Matias Cabral <matias.a.cabral@intel.com>
Now Open MPI requires a C99 compiler. Checking availability of
the following types is no more needed.
- `long long` (`signed` and `unsigned`)
- `long double`
- `float _Complex`
- `double _Complex`
- `long double _Complex`
Furthermore, the `#if HAVE_[TYPE]` style checking is not correct.
Availability of C types is checked by `AC_CHECK_TYPES` in `configure.ac`.
`AC_CHECK_TYPES` defines macro `HAVE_[TYPE]` as `1` in `opal_config.h`
if the `[TYPE]` is available. But it does not define `HAVE_[TYPE]`
(instead of defining as `0`) if it is not available. So even if we
need `HAVE_[TYPE]` checking, it should be `#if defined(HAVE_[TYPE])`.
I didn't remove `AC_CHECK_TYPES` for these types in `configure.ac`
since someone may use `HAVE_[TYPE]` macros somewhere.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
This fixes a regression introduced in commit open-mpi/ompi@f8318f0a8f.
Fixesopen-mpi/ompi#6069
Thanks Kawashima-san for the heads up !
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
ompi/mpiext/cuda/c/mpiext_cuda_c.h is automatically generated from
ompi/mpiext/cuda/c/mpiext_cuda_c.h.in at configure time.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
When the provider does not support FI_REMOTE_CQ_DATA, the OFI tag does not have
sizeof(int) bits for the rank. Therefore, unexpected behavior will occur when
this limit is crossed.
Check the max allowed number of ranks during add_procs() and return if there is
danger of exceeding this threshold.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
Commit 89da9651b inadvertantly #if'ed out both deprecated *and*
removed items from mpi.h. The intent was only to #if out items that
have been *removed* from the MPI specification and leave all items
that are merely deprecated.
This commit also re-orders the deleted typedef+functions to be in the
same order as they are listed in MPI-3.1 chapter 17, just to make
verifying/checking the code easier.
Note that --enable-mpi1-compatibility can still be used to restore
prototypes for the items that have been removed from the MPI
specification (e.g., MPI_Address()).
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
1. Remove debug output in iallgather (I have forgotten to remove it).
2. Remove an incorrect comment in description of ibcast
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
PR #5450 addresses MPI_IN_PLACE processing for basic collective algorithms.
But in conjunction with that, we need to check for MPI_IN_PLACE in tuned paths
as well before calling ompi_datatype_type_size() as otherwise we segfault.
MPI spec also stipulates to ignore sendcount and sendtype for Alltoall and
Allgatherv operations. So, extending the check to these algorithms as well.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
In order for this to work, error management needs to also be added to
NBC, from separate PR
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
The error field of requests needs to be rearmed at start, not at create
Signed-off-by: Aurelien Bouteiller <bouteill@icl.utk.edu>
using the standard $USER and $HOSTNAME environment variables
to make reproducible builds possible.
See https://reproducible-builds.org/ for why this is good.
This helps improve issue #3759
Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
Some compilers complain when comparing signed and unsigned. romio321
was doing just this. The check is meant to check whether a size (which
is an ADIO_Offset-- a signed number) will work with memcpy which takes
a size_t. To silence the warning I added a new type (ADIO_Size) which
is an unsigned type and cast the ADIO_Offset to this new type.
Fixes#5951
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
The monitoring PML hides it's existence from the OMPI infrastructure by
removing itself from the list of PML loaded components, remaining hidden
until MPI_Finalize.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
With this patch the best PML is selected earlier, before finalizing
the others PML. This provides a simpler mechanism to intercept and
highjack the PML (as done in the monitoring PML)
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
An implementation of R. Rabenseifner's algorithm for MPI_Iallreduce.
This algorithm is a combination of a reduce-scatter implemented with recursive vector halving
and recursive distance doubling, followed either by an allgather.
Limitations:
-- count >= 2^{\floor{\log_2 p}}
-- commutative operations only
-- intra-communicators only
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
check for providing a data representation that is actually supported
by ompio.
Add also one check for a non-NULL pointer in mpi/c/file_set_view
for the data representation.
Also fixes parts of issue #5643
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
Several fixes to string handling:
1. strncpy() -> opal_string_copy() (because opal_string_copy()
guarantees to NULL-terminate, and strncpy() does not)
2. Simplify a few places, such as:
* Since opal_string_copy() guarantees to NULL terminate, eliminate
some memsets(), etc.
* Use opal_asprintf() to eliminate multi-step string creation
There's more work that could be done; e.g., this commit doesn't
attempt to clean up any strcpy() usage.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
this ensures that all processes are done modifying a file
before syncing. Fixes an error in the testmpio testsuite.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
return MPI_ERR_ARG if the size of the fileview is not a
multiple of the size of the etype provided.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
return MPI_ERR_ACCESS if the user tries to read from a file
that was opened using MPI_MODE_WRONLY
return MPI_ERR_READ_ONLY if the user tries to write a file
that was opened using MPI_MODE_RDONLY
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
While we require C99 to build Open MPI, we do not require C99 to build
user MPI applications. As such, we shouldn't have C99-style comments
(i.e., "//"-style) in mpi.h.in.
Thanks to @AdamSimpson for reporting the issue.
This commit simply converts a //-style comment to a /**/-style
comment. No code or logic changes.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Implements recursive doubling algorithm for MPI_Iallgather.
The algorithm can be used only for power-of-two number of processes.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
Make sure all pending communications are done on all ranks before
closing the window. This way it will be safe to close the endpoints when
closing the component.
Signed-off-by: Yossi Itigin <yosefe@mellanox.com>
An implementation of R. Rabenseifner's algorithm for MPI_Ireduce.
This algorithm is a combination of a reduce-scatter implemented with recursive vector halving
and recursive distance doubling, followed either by a gather.
Limitations:
-- count >= 2^{\floor{\log_2 p}}
-- commutative operations only
-- intra-communicators only
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
OpenJDK 11 changed the default javadoc output HTML version to HTML 5
from HTML 4.01. It causes an error on building Open MPI configured
with `--enable-mpi-java` (default: disable). This fix is compatible
with older OpenJDK.
I don't know whether this problem exists with other vender's JDKs.
But this fix should be compatible with other JDKs because the new
syntax is used in other places in the same file.
Thanks to Siegmar Gross for the bug report.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The Open MPI code base assumed that asprintf always behaved like
the FreeBSD variant, where ptr is set to NULL on error. However,
the C standard (and Linux) only guarantee that the return code will
be -1 on error and leave ptr undefined. Rather than fix all the
usage in the code, we use opal_asprintf() wrapper instead, which
guarantees the BSD-like behavior of ptr always being set to NULL.
In addition to being correct, this will fix many, many warnings
in the Open MPI code base.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Fortran bindings were added to persistent collectives in 9e0115c980
but man was not updated.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Changes of nonblocking collectives in e98d794e8b and f750c6932c
are applied to persistent collectives.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
On OS X, where #pragma ident and #ident aren't supported, the
use of a static const star that was never used was generating
a warning (and, it should be noted, was useless, because the
compiler would optimize it away). Fix up the ident declaration
so that it is only created once in libmpi_mpifh.la.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Following the commit f750c6932c, I compared
`ompi/mpi/fortran/use-mpi-f08/*.F90` and
`ompi/mpi/fortran/use-mpi-f08/profile/p*.F90`, and
`ompi/mpi/fortran/use-mpi-f08/mod/mpi-f08-interfaces.F90` and
`ompi/mpi/fortran/use-mpi-f08/mod/pmpi-f08-interfaces.F90`.
There are many differences. Some are bugs of `MPI_*`, some are
bugs of `PMPI_*`. I'm not sure how these bugs affect applications.
To make it easy to compare these files future, I also removed
editorial differences.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
When --enable-mpi1-compatibility was specified, the ompi_mpi_ub/lb
symbols were #if'ed out of mpi.h. But the #defines for MPI_UB/LB
still remained. This commit also #if's out the MPI_UB/LB macros when
--enable-mpi1-compatibility is specified.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
The old/deprecated form of the file errhandler typedef used "fn" as a
suffix. The new form uses the name "function".
The MPI API typedef name has already been updated to use "function";
this commit updates the internal Open MPI typedef to use the name
"function" to match the MPI API name and avoid confusion.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Change default provider selection logic for the OFI MTL. The
old logic was whitelist-only, so any new HPC NIC provider would
have to ask users to do extra work or wait for an OMPI release
to be whitelisted. The reason for the logic was to avoid
selecting a "generic" provider like sockets or shm that would
frequently have worse performance than the optimized BTL options
Open MPI supports.
With the change, we blacklist the (small, relatively static) list
of providers that duplicate internal capabilities. Users can use
one of thse blacklisted providers in two ways: first, they can
explicitly request the provider in the include list (which will
override the default exclude list) and second, the can set a new
empty exclude list.
Since most HPC networks require special libraries and therefore
an explicit build of libfabric, it is highly unlikely that this
change will cause users to use libfabric when they didn't want to
do so. It does, however, solve the whitelisting problem.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Corrected the signatures of the collectives used by the Fortran 2008
interface to state correct intent for inout arguments and use the
ASYNCHRONOUS attribute in non-blocking collective calls.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Corrected the signatures of the collectives used by the Fortran 2008
interface to state correct intent for inout arguments and use the
ASYNCHRONOUS attribute in non-blocking collective calls. Also corrected
the C-bindings in Fortran accordingly
Signed-off-by: Philipp Otte <philipp.j.otte@googlemail.com>
Implements recursive doubling algorithm for MPI_Iexscan.
The algorithm preserves order of operations so it can be used both
by commutative and non-commutative operations.
The MCA parameter 'coll_libnbc_iexscan_algorithm' was added for dynamic
algorithm selection.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
Implements recursive doubling algorithm for MPI_Iscan. The algorithm preserves order of operations so it can be used both by commutative and non-commutative operations.
The MCA parameter coll_libnbc_iscan_algorithm was added for dynamic algorithm selection.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
- do not generate bindings for pompi_FOO_f symbols
(they are simply not used anywhere)
- move ompi_FOO_f bindings out of mpi_f08.mod into
ompi_mpifh_bindings.mod that is only used at build time
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
We previously needed to have empty targets because AM couldn't handle
having an AM_CONDITIONAL was targets in the "if" statement but not in the
"else". :-(
That now appears as an old automake bug that has been fixed,
so cleanup some Makefile.am
Thanks Jeff for the pointer.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
The parameter passed to NBC_Return_handle() was incorrectly casted
and not dereferenced.
Thanks Yossi for the bug report.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Move includes to top of file. Set some #defines so that
monitoring_prof.c compiles without warning (as identified by gcc 8 on
MacOS). Also ensure to include the internal Open MPI "mpi.h" file
(not some random system <mpi.h> file).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Gcc 8 identified hb_tree_csearch() as an infinite recursion, and it
turns out that we never call this function, anyway. So just remove
it.
Fixes#5670.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit updates the entire codebase to use specific opal types for
all atomic variables. This is a change from the prior atomic support
which required the use of the volatile keyword. This is the first step
towards implementing support for C11 atomics as that interface
requires the use of types declared with the _Atomic keyword.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
always initialize 'size'.
Only the a2a_sched_diss() alltoall algorithm is impacted,
and this algo is currently unused, so there is no need
to backport nor update the NEWS file for now.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
We need to return a persistent request.
`ompi_request_empty` is not a persistent request.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
file_delete triggers underneath the hood the full component selection
logic, since we do not have a file handle, just a file name.
As part of the selection logic, we have to however initiate the
framework-open of the fs component in case of ompio, since ompio
will call the delete function of the selected fs componentn, which
is based on the file system where the file is located.
This was not handled correctly so far. The problem however only
shows up if the first I/O operatin to be executed is a file_delete,
other wise the file_open will lead to the correct opening and initialization
of the fs framework. This commit ensures that we do the right thing
even if file_delete is the first file I/O operation in the application.
Fixes issue #5611
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
If an application opens a file for reading from multiple processes
using MPI_COMM_SELF (or another communicator that has distinct
process groups but the same comm-id, as can happen as the result
of comm_split), the naming chosen for the lockedfile or the mmapped
file used by the sharedfp/sm component would collide. This patch
ensures that the filename is different by integrating the process id
of rank 0 for each sub-communicator.
This fixes one aspect of the problem reported in github issue 5593
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
The aggregation code in osc/rdma is currently broken and will likely
not be reused. This commit cleans it out.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
When an application is not using multiple threads to call into MPI, we can
safely ask for FI_THREAD_DOMAIN setting from the provider as it should
translate to the least amount of locking in provider.
Conversely, for applications using THREAD_MULTIPLE, explicitly ask for
FI_THREAD_SAFE to prevent race conditions.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
gcc complains about ret possibly being used uninitialized. That will
never happen but we should still quiet the warning. This commit sets
ret to a valid value.
Fixes#5513
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
the sharedfp component has to be selected and opened before
we set the default file view during file_open. Otherwise
there is a sperious error message from the sharefp_file_seek
operation that is called during the file_set_view.
Fixes Issue #5560
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
Changes made:
- Create a new fs component for IME
- Create a new fbtl component for IME
- Modify the close function of OMPIO to finalize IME if necessary
Signed-off-by: Gaëtan Bossu <gbossu@ddn.com>
Signed-off-by: Sylvain Didelot <sdidelot@ddn.com>
Since progress entries array is globally allocated, it is susceptible
to race conditions when using multi-threaded applications. Allocating it
on the stack resolves any potential races as it is thread local by default.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
Having the "make_manpage.pl" script in the ompi/ tree broke
"./autogen.pl --no-ompi" (specifically: "make distcheck" of --no-ompi
builds would break).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
We were assuming that the array_of_indices has the same size as the
number of requests (incount), instead of the numberr of actually
active requests. While the patch is trivial, the question of the
size of the array_of_indices should be clarified in the MPI Forum.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
This commit updates the new custom matching code in pml/ob1 so it can
not be enabled with a configure option. This commit also renames the
fuzzy-matching headers to avoid potential name conflicts and removes
the use of C reserved identifiers.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
In the cleanup phase, it is possible for PtlMEUnlink() to return
PTL_IN_USE if the NIC is not done with the ME. This should not
be considered an error. This commit adds a retry loop around
PtlMEUnlink().
In some cases, the return value of PtlMEUnlink() and PtlCTFree()
was not checked at all. Check them with the same retry loop as
above.
Signed-off-by: Todd Kordenbrock <thkgcode@gmail.com>
When romio314 was first pulled in an extra patch was applied to it, see commit
92f6c7c1e2. Most of that patch is already present
in vanilla romio321, but the fix for MPIO_DATATYPE_ISCOMMITTED() isn't.
If that macro doesn't set err_ then some paths end up with a variable being used
uninitialized. In particular you can trace through romio321/romio/mpi-io/read.c
to see what happens with error_code. It's an uninitialized stack variable that goes
through three MPIO_CHECK_* macros none of which set it. The macros consistently set
error_code to a failure if they see something wrong, but they don't consistently
set it to success when things are fine.
And then in the last macro MPIO_CHECK_DATATYPE it tries to look at the value
of error_code that was never set.
Signed-off-by: Mark Allen <markalle@us.ibm.com>
The call of MPI_Allgatherv with sendbuf and sendtype parameters equal to MPI_IN_PLACE and NULL correspondingly, produces the segmentation fault.
The problem is that sendtype is used even when sendbuf value is MPI_IN_PLACE. But according to the standard, sendtype and sendcount parameters should be ignored in this case.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
-Updated blocking send to directly call functionality and
set completion events expected to 0 initally. This allows for optimization for
providers that support fi_tinject up to larger sizes. This also reduces
latency on running the OFI mtl with smaller sizes without requiring
calls to progress given fi_tinject is required to complete the messaging
before returning and will not create any events in the Completion Queue.
-Updated non-blocking send to directly call fi_tsend and avoid calling
fi_tinject as the functionality should not wait on completions. This
resolves a bug where applications calling MPI_Isend can overrun the
TX buffer with small (inject) messages causing a deadlock. In addition
this improves performance in message rates by preventing
waiting on any size message to complete in non-blocking send messages.
-Created common ompi_mtl_ofi_ssend_recv function to post the ssend recv
which is common between isend and send code paths.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
- If a message for a recv that is being cancelled gets completed after
the call to fi_cancel, then the OFI mtl will enter a deadlock state
waiting for ofi_req->super.ompi_req->req_status._cancelled which will
never happen since the recv was successfully finished.
- To resolve this issue, the OFI mtl now checks ofi_req->req_started
to see if the request has been started within the loop waiting for the
event to be cancelled. If the request is being completed, then the loop
is broken and fi_cancel exits setting
ofi_req->super.ompi_req->req_status._cancelled = false;
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
OFI providers may reserve some of the upper bits of the tag for
internal usage and expose it using mem_tag_format. Check for that
and adjust communicator bits as needed.
Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
- in sine cases persistent request was deleted during completion
callback, this cause double free of linked UCX request (assert
in debug build or hang in release build)
- UCX request is freed prior completion calback
Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
The call of MPI_Allgather with sendbuf and sendtype parameters equal to MPI_IN_PLACE and NULL correspondingly, produces the segmentation fault.
The problem is that sendtype is used even when sendbuf value is MPI_IN_PLACE. But according to the standard, sendtype and sendcount parameters should be ignored in this case.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
OMPIO now uses the correct delete function depending on the fs
mca_common_ompio_file_delete now works this way instead
of calling POSIX unlink:
- create a minimal file handle with the given file name
- select the best fs component using this file handle
- call the component-specific file delete function
Signed-off-by: Gaëtan Bossu <gbossu@ddn.com>
It is needed because the fs components might be queried due to a MPI_File_delete call.
And in this case, we don't have a communicator value.
Signed-off-by: Gaëtan Bossu <gbossu@ddn.com>
instead of invoking ompi_request_test_all(), that will end up
calling opal_progress() recursively, manually check the status
of the requests.
the same method is used in ompi_comm_request_progress()
Refs open-mpi/ompi#3901
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
- Added support in MTL_OFI_RETRY_UNTIL_DONE to handle -FI_EAGAIN
from the provider and correctly attempt to progress the OFI Completion
queue by calling ompi_mtl_ofi_progress.
- If events were pending that blocked OFI operations from being enqueued
they will be completed and the OFI operation will be retried once
ompi_mtl_ofi_progress has successfully completed.
- Updated MTL_OFI_RETRY_UNTIL_DONE to take a RETURN variable instead of
requiring the existance of a "ret" variable to pass back the return
value from completing the OFI operation.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
Blocking `MPI_SCATTER` and `MPI_SCATTERV` were fixed in 506d0e96f4
but noblocking `MPI_ISCATTER` and `MPI_ISCATTERV` were not fixed yet.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Move memory hooks init (for request based operation) in osc ucx to window
creation time, to avoid performance issue in MPI initialization.
Signed-off-by: Xin Zhao <xinz@mellanox.com>
This commit fixes a crash that occurs when using btl/vader as an RDMA
btl. This btl supports using CPU atomics and does not support using
the btl for self communication so we must use the local memory
optimizations in osc/rdma.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Data transferred by `MPI_BSEND` may corrupt if all of the following
conditions are met.
- The message size is less than the eager limit.
- The `btl_alloc` function in the BTL interface returns `NULL`
for some reason.
- The MPI program overwrites the send buffer after `MPI_BSEND`
returns.
The problem is in the way of pending a send request in ob1 PML.
The `mca_pml_ob1_send_request_start_copy` function retruns
`OMPI_ERR_OUT_OF_RESOURCE` if `mca_bml_base_alloc` function returns
`des = NULL`. In this case, the send request is added to the
`send_pending` list and `MPI_BSEND` returns immediately. Next time
the `mca_pml_ob1_send_request_start_copy` function tries sending,
the user buffer may have been overwritten by the MPI program.
Call hierarchy of `MPI_BSEND`:
```
MPI_Bsend
mca_pml_ob1_send
if (MCA_PML_BASE_SEND_BUFFERED == sendmode)
mca_pml_ob1_isend
MCA_PML_OB1_SEND_REQUEST_START_W_SEQ
mca_pml_ob1_send_request_start_seq
mca_pml_ob1_send_request_start_btl
if (size <= eager_limit)
if (req_send_mode == MCA_PML_BASE_SEND_BUFFERED)
mca_pml_ob1_send_request_start_copy
mca_bml_base_alloc
btl_alloc
if (OMPI_ERR_OUT_OF_RESOURCE == rc)
add_request_to_send_pending
ompi_request_free
```
To solve this problem, we should save the data to the buffer
attached by `MPI_BUFFER_ATTACH` before leaving `MPI_BSEND`.
This problem was introduced by ob1 optimization (commits 2b57f422
and a06e491c) in v1.8 series.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
This commit fixes an issue identified by MTT where we can have two
different sets of processes on the same node creating a shared memory
window with communicators sharing the same CID. To avoid this issue
the temporary filename now includes the creating processes vpid.
References #5363
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Use internal pack/unpack subroutines that operate on MPI_Aint instead of int
and hence solve some integer overflows.
Thanks Clyde Stanfield for reporting this issue.
Refs open-mpi/ompi#5383
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Current implementation of `coll/base/MPI_Scatter` is based on in-order binomial tree. This tree is right skewed and it provides good performance for a MPI_Gather operation. But for a MPI_Scatter operation left skewed binomial tree is effective.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
-Updated the design for sync send MPI calls to use 2 protocol bits for
denoting "sync_send" or "sync_send_ack".
-"Sync_send" is added to the send tag only and is masked out in receives
such that it can be read by the original Recv posted in the send/recv
operation.
-"Sync_send_ack" is sent from the recv callback to the send side. This 0
byte send does not generate a completion entry and instead sends the
message and immediately completes the opal completion in the recv.
-Tag formats ofi_tag_1 and ofi_tag_2 have been updated to include 2
more tag bits per format type due to the reduced protocal bits required
by OMPI.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
The call of MPI_Gather with sendbuf and sendtype parameters equal to MPI_IN_PLACE and NULL correspondingly, produces the segmentation fault in the root process.
The problem is that sendtype is used even when sendbuf value is MPI_IN_PLACE. But according to the standard (page 150, line 37), sendtype and sendcount parameters should be ignored in this case.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
- added common logging infrastructure for all
UCX modules
- all UCX modules are switched to new infra
Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
- some common functionality of del_procs calls is moved into
mca_common module
- blocking ucp_put call is replaced by non-blocking routine
Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
moving some code from fs/ufs into fs/base. The benefit of this approach is
that fs components that are fundamentally based on posix I/O (and only differ
in some non-posix functionality such as setting stripe size, or which hints
are being supported) can avoid having to replicate the same code over and
over again. First beneficiary is the lustre fs component, but more
are to follow soon.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
Intentionally do not mark some MPI-1 function pointer typedefs as
`__mpi_interface_removed__` because we have to use them in prototyping
some MPI-1 functions when `--enable-mpi1-compatibility` is used.
Fixes#5357.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit fixes a typo where a bcast is used instead of the intended
collective (barrier).
References #5262
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
