- Add some missing AC_CHECK_SIZEOF's in configure.ac
- Remove some unused variables
- Initialize some variables
- Fix some parameter types
- Cast where appropriate/safe to fix warnings
- Move ompi/mca/common/monitoring Fortran bindings to a separate .c
file so that they can use different #define's than the C bindings,
and therefore compile properly / without warnings.
- Fix signedness discrepancies
- Who knew? Separated these into multiple #if's, instead:
```
// This is undefined behavior
#define HAVE_FOO defined(FOO)
#define YOW (HAVE_FOO && defined(BAR))
```
- Fix some typos in OMPI_BUILD_HOST logic
- Don't "2>/dev/null" in OMPI_BUILD_HOST logic; it just hides errors
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
MacOS does not have "readlink -f" or "realpath", so use the
MacOS-provided Python, which we know has os.path.realpath().
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Abort early if the source directory, build directory, or prefix
contains spaces (either the supplied directory, or the canonicalized
version of the directory). This prevents users from getting cryptic
errors later in configure and/or make.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.
This patch moves the prrte submodule from the top-level to the
3rd-party directory, to match the behavior of other 3rd-party
packages like Libevent and PMIx. Since Open MPI does not
support building with an external PRRTE, that functionality
is skipped in this patch.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.
This patch moves the PMIx library bundled with Open MPI from a
MCA framework to a stand-alone library built outside of OPAL. Due
to the amount of code in the MCA base (and its assumptions about
being part of an MCA framework), the framework is left with no
active components. Any pre-installed version of PMIx 3.0.0 or
newer is preferred over the internal version.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.
This patch moves the hwloc library bundled with Open MPI from a
MCA framework to a stand-alone library built outside of OPAL. Due
to the amount of code in the MCA base (and its assumptions about
being part of an MCA framework), the framework is left with no
active components. Any pre-installed version of HWLOC 1.6 or
newer is preferred over the internal version.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.
This patch moves libevent from an MCA framework to a stand-alone
library built outside of OPAL. A wrapper in opal/util is provided
to minimize the unnecessary changes in the rest of the code. When
using the internal Libevent, it will be installed as a stand-alone
libevent.a, instead of bundled in OPAL. Any pre-installed version
of Libevent at or after 2.0.21 is preferred over the internal
version.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
With Open MPI 5.0, the decision was made to stop building 3rd-party
packages, such as Libevent, HWLOC, PMIx, and PRRTE as MCA components
and instead 1) start relying on external libraries whenever possible
and 2) Open MPI builds the 3rd party libraries (if needed) as
independent libraries, rather than linked into libopen-pal.
This patch is the first step in that process, providing foundational
changes required for supporting 3rd-party packages, such as changes
to autogen.pl, the top-level Makefile.am, and introducing two
Autoconf macros to support running sub-configure scripts; one
supporting source in tarball form and the other supporting
source in a sub-tree.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
This PR adds a new configure option: --with-wrapper-cc.
This option allows the user to set the compiler that will
be invoked by mpicc, shmemcc, etc. This allows the user
to build Open MPI with one compiler (a C standards compliant
compiler like clang or gcc) and have the wrapper use
another compiler (icc for example). This allows building
Open MPI with the best available compiler while still
supporting compiling Open MPI for a specific compiler
suite.
Signed-off-by: Nathan Hjelm <hjelmn@google.com>
1. We haven't used the -dlopen or -preopen options for years (if
ever?); no need for the `dlopen` LT_INIT option.
2. We haven't supported Windows for years; no need for the `win32-dll`
LT_INIT option.
Also, this commit includes a minor fix to a comment.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Also added infrastructure to have developers write man pages in
Markdown (vs. nroff). Pandoc >=v1.12 is used to convert those
Markdown files into actual nroff man pages.
Dist tarballs will contain generated nroff man pages; we don't want to
require users to have Pandoc installed. Anyone who builds Open MPI
from a git clone will need to have Pandoc installed (similar to how we
treat Flex). You can opt out of Open MPI's Pandoc-generated man pages
by configuring Open MPI with --disable-man-pages. This will also
disable "make dist" (i.e., "make dist" will error if you configured
with --disable-man-pages).
Also removed the stuff to re-generate man pages.
This commit also:
1. Includes a new man page, written in Markdown
(ompi/mpi/man/man5/MPI_T.5.md) that contains Open MPI-specific
information about MPI_T.
2. Includes a converted ompi/mpi/man/man3/MPI_T_init_thread.3.md (from
MPI_T_init_thread.3in -- i.e., nroff) just to show that Markdown
can be used throughout the Open MPI code base for man pages.
3. Made the Makefiles in ompi/mpi/man/man?/ be full-fledged
Makefile.am's (vs. Makefile.extras that are designed to be included
in ompi/Makefile.am). It is more convenient to test generation /
installation of man pages when you can "make" and "make install" in
their respective directories (vs. doing a build / install for the
entire ompi project).
4. Removed logic from ompi/Makefile.am that re-generated man pages if
opal_config.h changes.
Other man pages -- hopefully all of them! -- will be converted to
Markdown over time.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Found a handful of other URLs that weren't https-ized, so I updated
them, too (after verifying that they support https, of course).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
PRRTE needs hwloc and libevent, so it needs to be setup "late" in
configure.ac. However, we don't want to do it at the absolute bottom
of configure.ac, because right near the bottom, we setup CPPFLAGS (and
others) with values that are expected to be used only in
Makefile[.am]'s -- i.e., "$(foo)" values. Such values are not able to
be used here in configure.
Hence, move the PRRTE setup up above where we do these "final"/
only-relevant-to-Makefile[.am] CPPFLAGS (etc.) updates occur.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Add a framework to support different types of threading models including
user space thread packages such as Qthreads and argobot:
https://github.com/pmodels/argobotshttps://github.com/Qthreads/qthreads
The default threading model is pthreads. Alternate thread models are
specificed at configure time using the --with-threads=X option.
The framework is static. The theading model to use is selected at
Open MPI configure/build time.
mca/threads: implement Argobots threading layer
config: fix thread configury
- Add double quotations
- Change Argobot to Argobots
config: implement Argobots check
If the poll time is too long, MPI hangs.
This quick fix just sets it to 0, but it is not good for the
Pthreads version. Need to find a good way to abstract it.
Note that even 1 (= 1 millisecond) causes disastrous performance
degradation.
rework threads MCA framework configury
It now works more like the ompi/mca/rte configury,
modulo some edge items that are special for threading package
linking, etc.
qthreads module
some argobots cleanup
Signed-off-by: Noah Evans <noah.evans@gmail.com>
Signed-off-by: Shintaro Iwasaki <siwasaki@anl.gov>
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
This fixes a bug in the configuration system identified by a
change in the C++ standard. C++20 adds a new mandatory header
named version. This (and any system) header will always be
included with <> and not "". On case-insensitive filesystems
this new header conflicts with the VERSION file at the top
level of the build tree.
To fix this issue Open MPI needs to use -iquote instead of -I
for non-system include paths to ensure that these include are
only searched for the quote form of include. This commit also
adds a check to ensure that if the compiler does not support
-iquote that it falls back to -I until support can be added.
References #7155
Signed-off-by: Nathan Hjelm <hjelmn@google.com>
Looks like this was mistakently removed in the conversion to PRRTE.
We still need CFLAGS_WITHOUT_OPTFLAGS for Open MPI's MPI debugger
interface. Not having this functionality means that ompi/debuggers
was being compiled incorrectly, which led to -- among other things --
32 bit builds failing.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit contains the following changes:
The C++ bindings were removed from the standard in MPI-3.0. This
commit removes the entirety of the C++ bindings as well as the
support configury.
Removes all references to C++ from the man pages. This includes the
bindings themselves, all references to what C++ bindings return,
all not-available comments, and differences between C++ and other
language bindings.
If the user passes --enable-mpi-cxx, --enable-mpi-cxx-seek, or
--enable-cxx-exceptions, print a warning message an abort configure.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Signed-off-by: Nathan Hjelm <hjelmn@google.com>
Mark the --enable-orterun-prefix-by-default and
--enable-mpirun-prefix-by-default options as deprecated, but continue to
honor them by translating them to the new
--enable-prte-prefix-by-default option.
Signed-off-by: Ralph Castain <rhc@pmix.org>
We're dealing with CLI options that have been deleted, not
deprecated.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
(cherry picked from commit 5aea4446288db9207fbc60f23ee903b149217121)
Per the developer's meeting, add detection of the deprecated --with-pmi
(and its associated --with-pmi-libdir) configure option and error out
with a polite note of the change in support
Since "--with-pmi" now shows in the configure help output, mark the help
string with a giant *DEPRECATED* to warn users not to use it
Signed-off-by: Ralph Castain <rhc@pmix.org>
Ma
Signed-off-by: Ralph Castain <rhc@pmix.org>
If you autogen.pl --without-prrte, we wouldn't configure or build PRRTE
support. However, configuring with --disable-internal-rte wasn't working
as it was being ignored. This led to some false errors when compiling
with an earlier PMIx v2.2 release.
That said, there were a couple of places that needed protection against
PMIx v2.2.
Signed-off-by: Ralph Castain <rhc@pmix.org>
Will be replaced by PRRTE. Ensure that OMPI and OPAL layers build
without reference to ORTE. Setup opal/pmix framework to be static.
Remove support for all PMI-1 and PMI-2 libraries. Add support for
"external" pmix component as well as internal v4 one.
remove orte: misc fixes
- UCX fixes
- VPATH issue
- oshmem fixes
- remove useless definition
- Add PRRTE submodule
- Get autogen.pl to traverse PRRTE submodule
- Remove stale orcm reference
- Configure embedded PRRTE
- Correctly pass the prefix to PRRTE
- Correctly set the OMPI_WANT_PRRTE am_conditional
- Move prrte configuration to the end of OMPI's configure.ac
- Make mpirun a symlink to prun, when available
- Fix makedist with --no-orte/--no-prrte option
- Add a `--no-prrte` option which is the same as the legacy
`--no-orte` option.
- Remove embedded PMIx tarball. Replace it with new submodule
pointing to OpenPMIx master repo's master branch
- Some cleanup in PRRTE integration and add config summary entry
- Correctly set the hostname
- Fix locality
- Fix singleton operations
- Fix support for "tune" and "am" options
Signed-off-by: Ralph Castain <rhc@pmix.org>
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
When Slurm is built against PMIx, some installations place a copy of the
PMIx library that Slurm is linking against in the Slurm PMI location.
Current configury ignores that location. The desired behavior is to look
for a PMIx lib in that location when --with-pmi is given. If the user
also specifies --with-pmix and gives a different location, then override
anything previously found and look for it where the user directed.
Signed-off-by: Ralph Castain <rhc@pmix.org>
It never lived up to its purpose (and has caused amorphous indirect
errors such as https://github.com/open-mpi/ompi/issues/2519), so
delete it.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
The verbs and verbs_usnic components are now no longer necessary / no
longer used anywhere in the code base.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
`short float` support of the Intel C++ Compiler (group of C and C++
compilers), at least versions 18.0 and 19.0, is half-baked. It can
compile declarations of `short float` variables and expressions of
`sizeof(short float)` but cannot compile operations of `short float`
variables. In this situation, `AC_CHECK_TYPES(short float)` defines
`HAVE_SHORT_FLOAT` as 1 and compilation errors occur in
`ompi/mca/op/base/op_base_functions.c`. To avoid this error
tentatively, we disable `short float` support when using the Intel
C++ Compiler.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The type `short float`, which is proposed in ISO/IEC JTC 1/SC 22 WG 14
(C WG), is not supported by most compilers yet. But some compilers
(including gcc 7 for AArch64 and clang 6) support `_Float16`, which
is defined in ISO/IEC TS 18661-3:2015 (ISO/IEC JTC 1/SC 22/WG 14 N1945)
as an extensions for C. If it is detected in `configure`, it is used
as an alternate type of `short float` in Open MPI internal code.
This commit adds a `configure` option `--enable-alt-short-float=TYPE`.
It can be used to specify a type other than `short float` and `_Float16`
as the alternate type.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The type `short float` is proposed for the C language in ISO/IEC JTC
1/SC 22 WG 14 (C WG) for mainly IEEE 754-2008 binary16, a.k.a.
half-precision floating point or FP16.
By this commit, `short float` and `short float _Complex` are detected
in `configure` and used in Open MPI internal code. `MPI_SHORT_FLOAT`
and its complex number version are not added yet.
This commit changes values of existing `OPAL_DATATYPE_*` macros.
This change does not affect ABI compatibility of `libmpi.so` and the
like because these values are only used in OPAL and OMPI internal code.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
Now Open MPI requires a C99 compiler. Checking availability of
the following types is no more needed.
- `long long` (`signed` and `unsigned`)
- `long double`
- `float _Complex`
- `double _Complex`
- `long double _Complex`
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
The embedded PMIx Automake minimum version is already 1.13.4, so to
autogen.pl Open MPI successfully, you already have to have Autoamek
1.13.4. So we might as well make it official (i.e., bump Open MPI's
Automake minimum to match the Automake minimum in the embedded PMIx).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit updates the patcher component to either use the
__clear_cache intrinsic or the correct assembly to flush the
instruction cache.
Fixes#5631
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This code is the implementation of Software-base Performance Counters as described in the paper 'Using Software-Base Performance Counters to Expose Low-Level Open MPI Performance Information' in EuroMPI/USA '17 (http://icl.cs.utk.edu/news_pub/submissions/software-performance-counters.pdf). More practical usage information can be found here: https://github.com/davideberius/ompi/wiki/How-to-Use-Software-Based-Performance-Counters-(SPCs)-in-Open-MPI.
All software events functions are put in macros that become no-ops when SOFTWARE_EVENTS_ENABLE is not defined. The internal timer units have been changed to cycles to avoid division operations which was a large source of overhead as discussed in the paper. Added a --with-spc configure option to enable SPCs in the Open MPI build. This defines SOFTWARE_EVENTS_ENABLE. Added an MCA parameter, mpi_spc_enable, for turning on specific counters. Added an MCA parameter, mpi_spc_dump_enabled, for turning on and off dumping SPC counters in MPI_Finalize. Added an SPC test and example.
Signed-off-by: David Eberius <deberius@vols.utk.edu>
This patch disables the oshmem layer if there are no SPMLs that
will build. With the limited set of SPMLs available to support
oshmem, many builds end up installing an oshmem library that we
know will not work. There has been a bit of customer confusion
over oshmem, hopefully this will lead customers in the right
direction.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
Two related changes to allow projects to not build based on
configure test results, as opposed to only reacting to
user configure options today. Use case is disabling a project
like oshmem because no communication channels can be built.
First, Move PROJECT_* AM_CONDITIONALs from the top of configure to
the bottom, so that we can change the results during configure.
Second, add a DIST_SUBDIRS to Makefile.am (and populate it in
opal_mca) so that "make dist" will work even when a project is
disabled.
Signed-off-by: Brian Barrett <bbarrett@amazon.com>
The mpool/memkind component was using a deprecated "partitions" API.
This commit refactors the memkind component to make use of the
supported public API.
The public API uses 3 parameters to specify a mpool "kind":
- a memkind type (which for now is just default or HBM)
- a memkind policy
- a memkind_bits (partly to specify pagesize)
The MCA parameters were changed to reflect these memkind
parameters.
Add a make check test for sanity checking of the memkind component.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
We thought there was a silent data corruption issue on POWER 7/BE
systems, so we blocked building on POWER 7/BE systems altogether. We
later figured out that it was just data hangs -- not silent data
corruption. So in hindsight, the configure block probably wasn't
necessary -- but we didn't know it at the time.
Regardless, the hangs have now been fixed, and we're removing the
POWER 7/BE block in configure.
For more detail on the entire saga, see
https://github.com/open-mpi/ompi/issues/4349#issuecomment-374970982.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This commit updates the configure code for Open MPI to check for C11
support. The features requested are: atomics and thread local
storage.
References #3879
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
