1. Ensure to override CFLAGS properly. Move the setting of CFLAGS outside the AM_CONDITIONAL so that Automake doesn't get confused (because CFLAGS is already set inside an AM_CONDITIONAL -- moving it outside the conditional ensure that this local CFLAGS override trumps all other CFLAGS overrides).
2. Only build libfabric on Linux. Add a little more configury to ensure that we only try to build libfabric on Linux.
3. Remove a dead/unused file
4. Fix typo in condition check
5. Use "false", not "/bin/false"
This commit represents the conversion of the usnic BTL from verbs to
libfabric.
For the moment, libfabric is embedded in Open MPI (currently in the
usnic BTL). This is because the libfabric API is still changing, and
also has not yet been released. Ultimately, this embedded copy of
libfabric will likely disappear and the usnic BTL will rely on an
external installation of libfabric.
New configure options:
* --with-libfabric: will cause configure to fail if libfabric support
cannot be built
* --without-libfabric: will prevent libfabric support from being built
* --with-libfabric=DIR: use an external libfabric installation
* --with-libfabric-libdir=LIBDIR: when paired with --with-libfabric=DIR,
use LIBDIR for the libfabric installation library dir
The --with-libnl3[-libdir] arguments are now gone.
Some versions of gcc require this flag to be set before the __sync
builtin atomic compare and swap will support 128-bit values. If the
flag is required this check adds the flag to the CFLAGS.
A 128-bit compare-and-swap will enable a better atomic lifo implementation
that uses the pointer + counter method to avoid ABA issues. This commit
adds configury to check for the instruction (cmpxchg16b) and adds an
implementation that uses the __int128 type available in C99.
Clean up the orte_check_alps.m4. There was a little of
unnecesary stuff for handling cle 5, since it wasn't actually
doing the right thing, which would be to use pkg-config to
find dependencies both for dynamic and static linking.
Decouple the searching for alps libs, etc. from cray pmi.
Switch the alps ess and alps odls components' config files
to use the ALPS m4 macro.
alps configury fixes
Improve a check for detecting CLE release.
Improve an error message.
Currently only shows whether opal_mca.m4 decides whether to build a
component or not. This will be helpful in debugging a remote user's
setup that somehow, mysteriously decides not to build a specific
component.
Change the CPPFLAGS to just -I$srcdir, and change the include file to
check to be just opal/libltdl/ltdl.h. Do this because the CPPFLAGS
are passed to the preprocessor, and the C program that is passed
through the preprocessor is basically:
#include <opal/libltdl/ltdl.h>
Using the proper -I covers both VPATH and non-VPATH cases.
The mixing of the Slurm PMI and Cray PMI configure was getting
messy and dangerous - developers working on Slurm PMI often don't
have access to Cray PMI, etc.
This mod pulls out the Cray PMI configure into a separate m4 file.
Cray pmi is now configured as follows:
1) on Cray CLE 5 and higher, Cray PMI is auto detected. pkg-config
is used to resolve the necessary CPP flags, link flags and libs,
etc. Nothing needs to be added to the configure line to pick up
Cray PMI.
2) on legacy Cray CLE 4 systems with PMI 4.X, Cray PMI is also
auto detected.
3) on legacy Cray CLE 4 systems with PMI 5.X Cray PMI can't be auto-detected
owing to changes in the PMI pkg-config file which result in pkg-config
returning an error owing to a dependency of PMI on newer versions of ALPS
installs that are not present on CLE 4. So, for those falling in to this
situation, the --with-cray-pmi=(DIR) method needs to be used.
DIR specifies the Cray PMI install directory. The configure file looks
for required alps libraries first in /usr/lib/alps, then in
/opt/cray/xe-sysroot/default/usr/lib/alps.
In OMPI 1.8 and 1.8.1, we didn't allow gfortran to use the ignore-TKR
style "mpi" Fortran module (which is written in Fortran). Instead,
gfortran used the TKR style "mpi" module, which is written in C.
In OMPI 1.8.2, we removed the gfortran restriction (for suitably
modern gfortran -- i.e., >= 4.9). This allowed gfortran to use the
ignore-TKR style "mpi" module, but exposed a bug where MPI Fortran
sentinel values wouldn't be linked properly on OS X. The solution is
to use ```-Wl,-flat_namespace``` (see the OS X ld(1) man page for
details) when both building the libraries and in the wrapper
compilers.
Thanks to Github user yafshar for rasising the issue.
Fixes#259.
Per #257, the configure test used to check:
```c
```
But out in the code base, we actually do this:
```c
// ...
```
This commit updates the configure test to ```#define``` a string and
then use that with the various ident strategies.
Fixes#257
There is a continued interest in being able to do static
linking of executables. When using pkg-config to determine
linking requirements, the default behavior is to only
specify libraries required for dynamic linking.
In many cases, it suffices even for static linking to
use
pkg-config --libs package
to obtain the linkage string required for ld. However, for
libraries that have dependencies on non-standard libraries,
this may not be the case. The cray PMI library is such a case.
On a Cray CLE 5.X system, for example
pkg-config --libs cray-pmi
returns a string like
-L/opt/cray/pmi/5.0.2-1.0000.9906.117.2.ari/lib64 -lpmi
whereas
pkgconfig --static --libs cray-pmi
returns a string (modified for git commits) like
-L/opt/cray/pmi/5.0.2-1.0000.9906.117.2.ari/lib64
-L/opt/cray/ugni/5.0-1.0000.8563.255.10.ari/lib64
-L/opt/cray/alps/6.0.0-2.0000.8544.102.21.ari/lib64
-L/opt/cray/wlm_detect/1.0-1.0000.48964.8.2.ari/lib64
-L/opt/cray/rca/1.0.0-2.0000.49383.133.60.ari/lib64
-lpmi -lpthread -lalpslli -lpthread -lwlm_detect
-lugni -lpthread -lalpsutil -lpthread -lrca
Unfortunately the pkg.m4 does not contain a function for adding
the --static option to PKG_CHECK_MODULES. This commit defines
a new function, PKG_CHECK_MODULES_STATIC, which in addition to
defining the $1_CFLAGS and $1_LIBS variables defined by PKG_CHECK_MODULES,
also defines a $1_STATIC_LIBS.
We must also set the static library filenames properly (i.e., obey the
$FRAMEWORK_LIB_PREFIX that was set). Hence, set the prefixes earlier
in configure.ac, and update OPAL_MCA to set the static library names
properly.
These two macros set the MCA prefix and MCA cmd line id,
respectively. Specifically, MCA parameters will be named
PREFIX<foo> in the environment, and the cmd line will use
-ID foo bar.
These macros must be called during configure.ac and a value
supplied. In the case of Open MPI, the values given are
PREFIX=OMPI_MCA_ and ID=mca.
Other projects (such as ORCM) will call these macros with
their own unique values. For example, ORCM uses PREFIX=ORCM_MCA_
and ID=omca
This scheme is necessary to allow running Open MPI applications under
systems that use their own versions of ORTE and OPAL. For example,
when running OMPI applications under ORCM, we need the MCA params passed
to the ORCM daemons to be separated from those recognized by the OMPI application.
These two macros set the prefix for the OPAL and ORTE libraries,
respectively. Specifically, the OPAL library will be named
libPREFIXopen-pal.la and the ORTE library will be named
libPREFIXopen-rte.la.
These macros must be called, even if the prefix argument is empty.
The intent is that Open MPI will call these macros with an empty
prefix, but other projects (such as ORCM) will call these macros with
a non-empty prefix. For example, ORCM libraries can be named
liborcm-open-pal.la and liborcm-open-rte.la.
This scheme is necessary to allow running Open MPI applications under
systems that use their own versions of ORTE and OPAL. For example,
when running MPI applications under ORTE, if the ORTE and OPAL
libraries between OMPI and ORCM are not identical (which, because they
are released at different times, are likely to be different), we need
to ensure that the OMPI applications link against their ORTE and OPAL
libraries, but the ORCM executables link against their ORTE and OPAL
libraries.
This commit makes the folowing changes:
- Add support for the knem single-copy mechanism. Initially vader will only
support the synchronous copy mode. Asynchronous copy support may be added
int the future.
- Improve Linux cross memory attach (CMA) when using restrictive ptrace
settings. This will allow Open MPI to use CMA without modifying the system
settings to support ptrace attach (see /etc/sysctl.d/10-ptrace.conf).
- Allow runtime selection of the single copy mechanism. The default behavior
is to use the best available. The priority list of single-copy mehanisms is
as follows: xpmem, cma, and knem.
- Allow disabling support for kernel-assisted single copy.
- Some tuning and bug fixes.
