On x86_64 reading a 128-bit value requires multiple instructions.
Under some conditions if the counted pointer counter is read before
the item pointer the fifo can be left in an inconsistent state. This
commit forces the read of the counter to always be read first.
The fifo does not appear to suffer from the same race.
It is possible the compiler can reorder the read of the head item and
the head itself. This could lead to a situation where the item
returned was not really the head item.
Thanks to Nathan for pointing out that I missed snipping one line in
2f9c69f016 (I removed the trailing
comment, but not the trailing pragma -- oops!).
Retain the hetero-nodes flag for those cases where the user *knows* that there are differences and our automated system isn't good enough to see it.
Will obviously require further refinement as we find out which variances it can detect, and which it cannot.
1. Ensure to override CFLAGS properly. Move the setting of CFLAGS outside the AM_CONDITIONAL so that Automake doesn't get confused (because CFLAGS is already set inside an AM_CONDITIONAL -- moving it outside the conditional ensure that this local CFLAGS override trumps all other CFLAGS overrides).
2. Only build libfabric on Linux. Add a little more configury to ensure that we only try to build libfabric on Linux.
3. Remove a dead/unused file
4. Fix typo in condition check
5. Use "false", not "/bin/false"
This commit represents the conversion of the usnic BTL from verbs to
libfabric.
For the moment, libfabric is embedded in Open MPI (currently in the
usnic BTL). This is because the libfabric API is still changing, and
also has not yet been released. Ultimately, this embedded copy of
libfabric will likely disappear and the usnic BTL will rely on an
external installation of libfabric.
New configure options:
* --with-libfabric: will cause configure to fail if libfabric support
cannot be built
* --without-libfabric: will prevent libfabric support from being built
* --with-libfabric=DIR: use an external libfabric installation
* --with-libfabric-libdir=LIBDIR: when paired with --with-libfabric=DIR,
use LIBDIR for the libfabric installation library dir
The --with-libnl3[-libdir] arguments are now gone.
This commit adds a new class: opal_fifo.h. The new class has atomic, non-atomic,
and opal_using_threads() conditoned routines. It should be used when first-in
first-out is required and should perform much better than using locks and an
opal_list_t. Like with opal_lifo_t there are two versions of the atomic
implementation: 128-bit compare-and-swap, and spin-locked. More implementations
can be added later (LL/SC comes to mind).
This commit also adds a unit test for the opal_fifo_t class. This test verifies
the fifo implementation when using multiple threads.
- Rename opal_atomic_lifo_t to opal_lifo_t to reflect both atomic and
non-atomic usage. Added new routines (opal_lifo_*_st) for non-atomic
usage as well as routines conditioned off opal_using_threads(). The
atomic versions are always thread safe and the non-atomic are always
not thread safe.
- Add a new atomic lifo implementation that makes use of 128-bit
compare-and-swap. The new implementation should scale better with
larger numbers of threads.
- Add threading unit test for opal_lifo_t.
There currently is no standard support for 128-bit integer types. Any use
of the __int128 and int128_t types can lead to warnings from the compiler
when using -Wpedantic. Additionally, some compilers may support __int128
and other may support int128_t. This commit addresses both issues by
defining opal_int128_t if there is a supported 128-bit type. In the
case of GCC a pragma has been added to suppress warnings about __int128
not being a standard C type.
A 128-bit compare-and-swap will enable a better atomic lifo implementation
that uses the pointer + counter method to avoid ABA issues. This commit
adds configury to check for the instruction (cmpxchg16b) and adds an
implementation that uses the __int128 type available in C99.
If OPAL_MODEX_RECV() returns OPAL_ERR_NOT_FOUND, the peer didn't
send any Portals4 BTL info. This is not a fatal error. Instead of
disqualifying the Portals4 BTL just ignore that peer.
@jsquyres reported this in #194.
