Identify the index of each aggregator process in order to restrict the call to write_init function by the specific aggregator.
Signed-off-by: raafatfeki <fekiraafat@gmail.com>
Instead of using a temporary buffer and copy data into the temp buffer before sending, use a derived datatype to describe the data that needs to be sent during a cycle in the collective I/O operation.
Signed-off-by: raafatfeki <fekiraafat@gmail.com>
import of the new vulcan component. It is an enhanced version
of the two_phase component, which uses however the ompio internal
codes/loops to assemble the data arrays. It is therefore more inline
with the dynamic and dynamic_gen2 component, and will be easier to
maintain.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
If a subroutine of the Fortran `use-mpi-f08` binding in an MPI extension
have a `LOGICAL` parameter and no `TYPE(MPI_Status)` parameter,
it needs to use the `mpi_ext` module and call its corresponding subroutine
in the `mpif-h` directory, as explained in
`ompi/mpi/fortran/use-mpi-f08/mpi-f-interfaces-bind.h`.
However, as shown in the figure below, the required directories are dependent
on each other, and "Can't open module file" error occurs at build time.
ompi/mpiext/{extension name}/use-mpi-f08
A |
| |
| V
ompi/mpi/fortran/use-mpi-f08 <--- ompi/mpi/fortran/mpiext (mpi_ext.mod)
In order to solve this problem, change the configuration and the build order.
- divide Fortran extension directory (`ompi/mpi/fortran/mpiext`)
into the directories for `use-mpi` and for `use-mpi-08`
- `ompi/mpi/fortran/mpiext-use-mpi` : for `use-mpi` (mpi_ext.mod)
- `ompi/mpi/fortran/mpiext-use-mpi-f08` : for `use-mpi-08` (mpi_f08_ext.mod)
- change to the following build order about Fortran `use-mpi` and
`use-mpi-f08` bindings in `ompi`
1. mpi_ext bindings of MPI extensions (`mpiext/{extension name}/use-mpi` directory)
2. Fortran use-mpi (`mpi/fortran/use-mpi-[ignore-]tkr` directory)
3. Fortran extension for use-mpi (`mpi/fortran/mpiext-use-mpi` directory)
4. Fortran use-mpi-f08 modules only (`mpi/fortran/use-mpi-f08/mod` directory)
5. mpi_f08_ext bindings of MPI extensions (`mpiext/{extension name}/use-mpi-f08` directory)
6. Fortran use-mpi-f08 (`mpi/fortran/use-mpi-f08` directory)
7. Fortran extension for use-mpi-f08 (`mpi/fortran/mpiext-use-mpi-f08` directory)
Signed-off-by: Kurita, Takehiro <fj6370fp@aa.jp.fujitsu.com>
There was a race condition in 35438ae9b5: if multiple threads invoked
ompi_mpi_init() simultaneously (which could happen from both MPI and
OSHMEM), the code did not catch this condition -- Bad Things would
happen.
Now use an atomic cmp/set to ensure that only one thread is able to
advance ompi_mpi_init from NOT_INITIALIZED to INIT_STARTED.
Additionally, change the prototype of ompi_mpi_init() so that
oshmem_init() can safely invoke ompi_mpi_init() multiple times (as
long as MPI_FINALIZE has not started) without displaying an error. If
multiple threads invoke oshmem_init() simultaneously, one of them will
actually do the initialization, and the rest will loop waiting for it
to complete.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Implements butterfly algorithm for MPI_Reduce_scatter.
The algorithm can be used both by commutative and non-commutative operations, for power-of-two and non-power-of-two number of processes.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
Get the OMPI rte/pmix component working. This was tested using PRRTE as the RM, configuring OMPI using:
* autogen --no-orte
* with external libevent, external hwloc, and external PMIx master
* configuring PMIx master with the same libevent and hwloc
* execute the application using PRRTE's "prun" launcher, which has the same cmd line as ORTE's mpirun
Note that PMIx master appears to have a bug in the event notification system that caches job termination events. Thus, the first execution runs fine, but subsequent executions cause an "abort" when the OMPI default error handler is invoked upon notification of the prior job's termination. Will work that separately.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 134cca9ac0de092d767999357573a31703f72292)
This checkin mainly concerns our internal info keys that are registering
for callbacks via opal_infosubscribe_subscribe(). Those keys need to have
an extra __IN_<key>/val stored to preserve their pre-callback value. So
that means our internal keys are limited to 5 chars shorter than the usual
key length limit.
The code previously would have been silently inactive if a large key happened
to come in, now it warns and also uses snprintf() to avoid compiler warnings.
I'm also making the top-level MPI_Info_set warn if the user uses our reserved
"__IN_" prefix. I had wanted the feature to be more invisible than that, but
it would require a more sophisticated approach to change that.
Signed-off-by: Mark Allen <markalle@us.ibm.com>
Per MPI-3.1:8.7.1 p361:11-13, it's valid for MPI_FINALIZED to be
invoked during an attribute destruction callback (e.g., during the
destruction of keyvals on MPI_COMM_SELF during the very beginning of
MPI_FINALIZE). In such cases, MPI_FINALIZED must return "false".
Prior to this commit, we hung in FINALIZED if it were invoked during
a COMM_SELF attribute destruction callback in FINALIZE. See
https://github.com/open-mpi/ompi/issues/5084.
This commit converts the MPI_INITIALIZED / MPI_FINALIZED
infrastructure to use a single enum (ompi_mpi_state, set atomically)
to represent the state of MPI:
- not initialized
- init started
- init completed
- finalize started
- finalize past COMM_SELF destruction
- finalize completed
The "finalize past COMM_SELF destruction" state is what allows us to
return "false" from MPI_FINALIZED before COMM_SELF has been fully
destroyed / all attribute callbacks have been invoked.
Since this state is checked at nearly every MPI API call (to see if
we're outside of the INIT/FINALIZE epoch), care was taken to use
atomics to *set* the ompi_mpi_state value in ompi_mpi_init() and
ompi_mpi_finalize(), but performance-critical code paths can simply
read the variable without needing to use a slow call to an
opal_atomic_*() function.
Thanks to @AndrewGaspar for reporting the issue.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
for very large offsets, ome ariables used in the fcoll/dynamic_gen2
code base were under certain circumstances not large enough to hold
intermediate values. This issue was more detected in the vulcan component
but could happen in the dynamic_gen2 component as well.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
appends MPI1 compatible source files instead of redefining all the source files
fix a typo from open-mpi/ompi@89da9651bb
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
This commit adds a new configure option: --enable-mpi1-compat. Without
this option we will no longer provide APIs, typedefs, and defines that
were removed from the standard in MPI-3.0. This option will exist for
one major release (Open MPI v4.x.x) and then the option and associated
code will be removed in Open MPI v5.x.x. Open MPI has already
internally prepared for this change. Please prepare your codes
accordingly.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit adds a new MCA variable to set the location of the backing
store: osc_sm_backing_directory. The default on Linux has been
changed to use /dev/shm to improve performance in cases where /tmp is
not a tmpfs.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit adds a new MCA variable to set the location of the backing
store: osc_rdma_backing_directory. The default on Linux has been
changed to use /dev/shm to improve performance in cases where /tmp is
not a tmpfs.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Per discussion at
https://github.com/open-mpi/ompi/issues/2614#issuecomment-392815654,
do not allow for selection of the OSC PT2PT when creating an MPI RMA
window when THREAD_MULTIPLE is active. Print a helpful message and
return a not-supported error.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit d0ffd660841623c02d1dfa3151e7f7afd3327698)
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Implements butterfly algorithm for MPI_Reduce_scatter_block.
The algorithm can be used both by commutative and non-commutative
operations, for power-of-two and non-power-of-two number of processes.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
Remove the MXM MTL, which has been deprecated in preference for
the Yalla PML. This was discussed at the last developers meeting
and somehow I ended up with the action item to do the removal.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
The Java configury is split into two parts:
1. Determine if we want MPI Java bindings.
2. Find the Java compiler (and related).
This commit does a few things:
- Move the "Find the Java compiler" step from OPAL to OMPI (because
there is no Java in OPAL, and there doesn't appear to be any
immanent danger that there will be).
- As a direct consequence, remove the --enable-java CLI option
(--enable-mpi-java still remains). Enabling the MPI Java bindings
and enabling Java are now considered the same thing (since there
is no Java elsewhere in the code base, the different was
meaningless).
- Only invoke the "Find the Java compiler" step if we actually want
the MPI Java bindings.
- A few miscellaneous Java-related cleanups in configury (E.g., change
testing "$foo" == "1" to $foo -eq 1, etc.
This commit is mostly s/opal/ompi/gi in many places in configury and
shifting code around. But it looks bigger than it actually is because
of two reasons:
1. Some files were renamed:
* ompi_setup_java.m4 -> ompi_setup_mpi_java.m4 (setup MPI Java bindings)
* opal_setup_java.m4 -> ompi_setup_java.m4 (setup Java compiler)
2. Indenting level changed in (the new) ompi_setup_java.m4.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
- rename ompi_coll_base_reduce_scatter_block_basic to
more self descriptive ompi_coll_base_reduce_scatter_block_basic_linear
- fix the description of the coll_tuned_reduce_scatter_block_algorithm
MCA param
this fixes and documents previous open-mpi/ompi@0e8b35b615
MPI_Reduce_scatter_block used to be implemented by the coll/basic module only.
A new algo (recursive doubling) was recently introduced and can be used via the coll/tuned module,
but we never intended to make it the default algo.
In order to "restore" the previous default, the initial algo was moved from coll/basic to coll/base,
and is now used by default by coll/tuned.
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
The MPI spec defines that the "mpi" and "mpi_f08" module Fortran
bindings support passing by parameters by name. Hence, we need to use
the MPI-spec-defined parameter names ("dummy variables", in Fortran
parlance) for the "mpi" and "mpi_f08" modules.
Specifically, Fortran allows calls to procedures to be written with
keyword arguments, e.g., "call mpi_sizeof(x=x,size=rsize,ierror=ier)"
An "explicit interface" for the procedure must be in scope for this to
be allowed in a Fortran program unit. Therefore, the explicit
interface blocks we provide in the "mpi" and "mpi_f08" modules must
match the MPI published standard, including the names of the dummy
variables (i.e., parameter names), as that is how Fortran programs may
call them.
Note that we didn't find this issue previously because even though the
MPI spec *allows* for name-based parameter passing, not many people
actually use it. I suspect that we might have some more incorrect
parameter names -- we should probably do a full "mpi" / "mpi_f08"
module parameter name audit someday.
Thanks to Themos Tsikas for reporting the issue and supplying the
initial fix.
Signed-off-by: themos.tsikas@nag.co.uk
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
The files array was also storing $phase.prof. This was leading to
$phase.prof's output getting dumped into itself again and again. Updated
code to initialise files array with files other than $phase.prof.
Signed-off-by: Ninad Prabhukhanolkar <ninadchess96@gmail.com>
Set mpi.lo dependencies outside of AM conditionals. Per
https://github.com/open-mpi/ompi/issues/5085, make mpi.lo depend on
whatever we decide the sizeof source files are (which may be empty, if
this compiler does not support the Right Stuff for MPI_SIZEOF, or it
may be mpi-tkr-sizeof.[h|f90]).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
whenever possible.
Add the missing OMPI_FORTRAN_BUILD_SIZEOF macro to Fortran
and add a missing dependency.
Thanks Themos Tsikas for reporting this issue.
Refs open-mpi/ompi#5085
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
keep track of the sizeof the blocklen_per_process and displs_per_process
on the aggregator datastructure to minimze the number of realloc function
calls required in the shuffle_init operation.
Signed-off-by: raafatfeki <fekiraafat@gmail.com>
This commit attempts to update the romio io component to not use
functions removed in MPI-3.0 (2012). This is a first cut and will
probably need to be reviewed for correctness.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
romio assumes that all predefined datatypes are contiguous. Because of
the (terribly named) composed datatypes MPI_SHORT_INT, MPI_DOUBLE_INT,
MPI_LONG_INT, etc this is an incorrect assumption. The simplest way to
fix this is to override the MPI_Type_get_envelope and
MPI_Type_get_contents calls with calls that will work on these
datatypes. Note that not all calls to these MPI functions are
replaced, only the ones used when flattening a non-contiguous
datatype.
References #5009
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Fixes issue #5069, which relates a BigMPI bug with the use of
MPI_Type_vectpor to construct very large datatypes (>2GB).
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
This commit fixes a segfault in mtl-portals4 finalize(). The segfault
occurs if finalize() is called without any calls to add_procs(). This
commit resolves the segfault by skipping the progress() loop in
finalize() if the Portals was not initialized.
Signed-off-by: Todd Kordenbrock (thkgcode@gmail.com)
Per 0ab6b201fe, note in the MPI_Comm_spawn_multiple.3in man page that
the array_of_commands does not need to be terminated -- it just need
to have exactly "count" entries. In the Fortran binding, at least,
this is different than in prior released versions of Open MPI (it's
not a backwards incompatibility, since prior versions of Open MPI
required array_of_commands to be blank-string-terminated in Fortran --
this change makes Open MPI be *less* restrictive, and therefore still
backwards compatible).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
MPI defines the "argv" param to Fortran MPI_COMM_SPAWN as being
terminated by a blank string. While not precisely defined (except
through a non-binding example, Example 10.2, MPI-3.1 p382:6-29), one
can infer that the "array_of_argv" param to Fortran
MPI_COMM_SPAWN_MULTIPLE is also a set of argv, each of which are
terminated by a blank line.
The "array_of_commands" argument to Fortran MPI_COMM_SPAWN_MULTIPLE is
a little less well-defined. It is *assumed* to be of length "count"
(another parameter to MPI_COMM_SPAWN_MULTIPLE) -- and *not* be
terminated by a blank string. This is also given credence by the same
example 10.2 in MPI-3.1.
The previous code assumed that "array_of_commands" should also be
terminated by a blank line -- but per the above, this is incorrect.
Instead, we should just parse our "count" number of strings from
"array_of_commands" and *not* look for a blank line termination.
This commit separates these two cases:
* ompi_fortran_argv_blank_f2c(): parse a Fortran array of strings out
and stop when reaching a blank string.
* ompi_fortran_argv_count_f2c(): parse a Fortran array of strings out
and stop when "count" number of strings have been parsed.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
javah is no more available from Java 10, so try
javac -h first (available since Java 8) and fallback on javah
Refs. open-mpi/ompi#5000
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
the ompio module resets the amode from WRONLY to RDWR in order
to accoomodate data sieving in the two-phase fcoll componet. This
leads however to an error if MPI_MODE_SEQUENTIAL has been requested
by the user, since MODE_SEQUENTIAL is incompatible with MODE_RDWR.
SInce the change to the amode was done after opening the file for
individual file pointers but before opening the file for shared filepointers,
this lead to an error message in the sharedfp component.
Note, that data sieving is never necessary if MODE_SEQUENTIAL is set,
so this should not be a problem for any scenario.
Fixes#4991
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
instead of using a temporary buffer and copy data into the temp buffer before sending, use a derived datatype to describe the data that needs to be sent during a cycle in the collective I/O operation.
Signed-off-by: raafatfeki <fekiraafat@gmail.com>
Implements recursive doubling algorithm for MPI_Scan and MPI_Exscan.
The algorithm preserves order of operations so it can be used both
by commutative and non-commutative operations.
Signed-off-by: Mikhail Kurnosov <mkurnosov@gmail.com>
This commit fixes a bug is osc/rdma that can occur if the total size
of the shared memory segment gets larger than 4 GiB. The bug was
caused by a typo. The type of my_base_offset should have been size_t
not int.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This will almost certainly never happen, but be defensive and
guarantee that we never return an uninitialized variable.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Ensure to initialized ret_code. This problem will likely never occur
in practice, but we might as well be defensive about it.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
The various RMA functions need to have the asynchronous property on
all buffers. This property was missing and some buffers were
incorrectly marked as intent(in). This commit fixes the function
signatures.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
somehow the flag indicating to gather performance data
on collective io operations has changed to 1 accidentally.
Should be 0 ( false) by default.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
never got to move this sharedfp component into anything
usable. Can easily be restored if necessary.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
plfs components are at this point not utilized by anybody as far as I know.
Easy to bring back if we want to.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
This commit is a large update to the osc/rdma component. Included in
this commit:
- Add support for using hardware atomics for fetch-and-op and single
count accumulate when using the accumulate lock. This will improve
the performance of these operations even when not setting the
single intrinsic info key.
- Rework how large accumulates are done. They now block on the get
operation to fix some bugs discovered by an IBM one-sided test. I
may roll back some of the changes if the underlying bug in the
original design is discovered. There appear to be no real
difference (on the hardware this was tested with) in performance so
its probably a non-issue. References #2530.
- Add support for an additional lock-all algorithm: on-demand. The
on-demand algorithm will attempt to acquire the peer lock when
starting an RMA operation. The lock algorithm default has not
changed. The algorithm can be selected by setting the
osc_rdma_locking_mode MCA variable. The valid values are two_level
and on_demand.
- Make use of the btl_flush function if available. This can improve
performance with some btls.
- When using btl_flush do not keep track of the number of put
operations. This reduces the number of atomic operations in the
critical path.
- Make the window buffers more friendly to multi-threaded
applications. This was done by dropping support for multiple
buffers per MPI window. I intend to re-add that support once the
underlying performance bug under the old buffering scheme is
fixed.
- Fix a bug in request completion in the accumulate, get, and put
paths. This also helps with #2530.
- General code cleanup and fixes.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
data sieving has to occur for any offset provided that is larger
or equal zero for this implementation to work correctly.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
Allowing MPI_PROC_NULL as a neighbor in any topology allows us to add
gaps on the send and recv buffers. This does make the traditional
neighbor collective have a similar behavior as the V version, but in
same time it allows the users to skip the step where they prepare the
counts and the displacement array.
For more info please take a look at issue #4675.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
this commit fixes an issue observed with romio314 and the hdf5 1.10.x testsuite.
The ADIOI_Datatype_iscontig() routine in romio314/src/io_romio314_module.c
will now return for a datatype of size 0 that it is contiguous, even if the extent
of the datatype is non-zero. This avoids a segmentation fault observed in the
ADIOI_Flatten routine, and fixes this particular with the hdf5 1.10.x testsuite in
OpenMPI with romio314.
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
As discussed on https://github.com/mpi-forum/mpi-issues/issues/77#issuecomment-369663119
the conversion to double in the MPI_Wtime decrease the range
and accuracy of the resulting timer. By setting the timer to
0 at the first usage we basically maintain the accuracy for
194 days even for gettimeofday.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
This commit fixes#4795
- Fixed typo that sometimes causes deadlock in change of protocol.
- Redesigned out of sequence ordering and address the overflow case of
sequence number from uint16_t.
Signed-off-by: Thananon Patinyasakdikul <tpatinya@utk.edu>
For some of our configuration this flag increases per-process contribution
by ~20% while it is not being used currently.
The consumer of this flag was communicator ID calculation logic, but it was
changed in 0bf06de3f1.
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
-Updated ompi_mtl_ofi_progress to use an array to read CQ events up to a
threshold that can be set by the Open MPI User.
-Users can adjust the number of events that can be handled in the
ompi_mtl_ofi_progress by setting "--mca mtl_ofi_progress_event_cnt #".
-The default value for the the number of CQ events that can be read in a
single call to ofi progress is 100 which is an average
based off workload usecase anaylsis showing 70-128 as the range of
multiple events returned during ofi progress.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
This commit fixes a flaw in the progress function for libnbc. The
function was unconditionally taking a lock even if there are no
requests to process. This lock was showing up in vtune traces of
multi-threaded benchmarks.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
after performing the final OBJ_RELEASE on the request,
reset the user level variable to MPI_REQUEST_NULL.
Otherwise the c_2_f translation step in the fortran
interface fails.
Fixes issue #4807
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
This commit fixes a flaw in the eager limit check in pml/ob1. The
check was incorrectly checking if RDMA-only BTLs (BTLs without the
send flag) has a valid eager limit. This commit fixes the check by
adding an additional check for the send flag on the BTL module.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
The osc monitoring component needed to include other OSC components
header in order to be able tu access communicator through the
component specific ompi_osc_*_module_t structures. This commit remove
the dependency, and resolve the issue #4523.
Extend the common monitoring API.
* Now it's possible to translate from local rank to world rank from
both the communicator and the group.
* Remove useless hashtable as we directly use the w_group contained
in window structure.
Add automatic generation at config time.
The templates are expanded at configure time. It creates a new header
file that generates all the variables/functions needed. Adding this
during the autogen automagicaly generates for each of the available
modules the proper functions.
Only keep a generated argv-style array.
Following Jeff's advice, the configure.m4 file generate a simple array
of module variables to be iterated over to find the proper module.
Signed-off-by: Clement Foyer <clement.foyer@inria.fr>
If component selection fails, then module->bases might be unallocated
when ompi_osc_sm_free() in invoked, so test it before trying to free()
module->bases[0].
Thanks Martin Binder for the report.
Refs open-mpi/ompi#4770
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
an erroneous return statement has creeped in commit 1885d99
which leads to some processes not resetting stripe_size
and stripe_count correctly. This can lead in 3.0.x to different
fcoll modules being selected. The impact is not that dramatic on
master and 3.1.x, but could lead to problems as well.
Fixes#4745
Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
