1
1

5576 Коммитов

Автор SHA1 Сообщение Дата
Jeff Squyres
14563770a1 btl/usnic: amend Makefile.am fix from b4097626ab
Use $(AM_CPPFLAGS) in $(usnic_btl_run_tests_CPPFLAGS) so that we don't
have to replicate hard-coded values.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-03-05 09:30:21 -08:00
Nysal Jan K.A
da6d038d13 opal/asm: Remove an unused variable
Signed-off-by: Nysal Jan K.A <jnysal@in.ibm.com>
2019-03-05 19:12:19 +05:30
Nysal Jan K.A
5ffde16d9b opal/asm: Fix a compiler warning on POWER arch
OPAL_XLC_INLINE_ASSEMBLY was removed in commit ebce88b7ad.
Removing dead code, which also fixes a compiler warning.

Signed-off-by: Nysal Jan K.A <jnysal@in.ibm.com>
2019-03-05 17:03:53 +05:30
Gilles Gouaillardet
b4097626ab btl/usnic: fix usnic_btl_run_tests CPPFLAGS
do define the OMPI_LIBMPI_NAME macro via the CPPFLAGS.
The issue occurs when Open MPI is configured with
--enable-opal-btl-usnic-unit-tests

Thanks George Marselis for reporting this issue

Refs. open-mpi/ompi#6441

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2019-03-05 09:57:55 +09:00
Ralph Castain
8fd6107987
Merge pull request #6418 from rhc54/topic/slurm
Update slurm pmi configury to account for pmix
2019-02-27 14:30:45 -08:00
Ben Menadue
17dcc7041a Hold off running hwloc:external feature tests until after we decide if we're using the internal or external component. This fixes #6430.
Signed-off-by: Ben Menadue <ben.menadue@nci.org.au>
2019-02-25 16:58:11 +11:00
Ralph Castain
cd1b5641be Update slurm pmi configury to account for pmix
When Slurm is built against PMIx, some installations place a copy of the
PMIx library that Slurm is linking against in the Slurm PMI location.
Current configury ignores that location. The desired behavior is to look
for a PMIx lib in that location when --with-pmi is given. If the user
also specifies --with-pmix and gives a different location, then override
anything previously found and look for it where the user directed.

Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-21 11:33:35 -08:00
Artem Polyakov
13a8e42108
Merge pull request #6163 from artpol84/osc/mt_submission
Refactoring of osc/ucx component for MT
2019-02-20 09:41:27 -08:00
Jeff Squyres
170d5d119e
Merge pull request #6409 from dmitrygladkov/topic/btl/tcp
btl/tcp: Fix copy-paste misprint
2019-02-20 12:12:18 -05:00
Dmitry Gladkov
9920da4992 btl/tcp: Fix copy-paste misprint
Signed-off-by: Dmitry Gladkov <dmitrygla@mellanox.com>
2019-02-20 11:18:02 +02:00
Artem Polyakov
91d6115d99 opal/common/ucx: Adjust the threasholds for periodical flushes
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
3aadc2b5e1 opal/common/ucx: Fix periodical flush in the worker pool
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
84dfe1277c opal/common/ucx: Rename wpool recv_worker to dflt_worker
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
8a990c2b64 opal/common/ucx: Add comments clarifying data structures
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
19e2ae2efb opal/common/ucx: Switch to opal/tsd
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
7984d7d997 opal/common/ucx: Remove unused debugging macro
Will be reintroduced later if needed and after adaptation to the OMPI
infrastructure.

Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
43f16d8796 opal/common/ucx: Remove common_ucx_int.h
Place the content of common_ucx_int.h back to the common_ucx.h and
include common_ucx_wpool.h explicitly.

Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
bb7d360621 opal/common/ucx: add refcnt in tlocal_ctx_tbl entry to keep track of usage
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
101036651b opal/common/ucx: Fix the bug in wpool's periodical flush
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
bcb52ecade opal/common/ucx: add winfo ptr into req
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
33517428a1 opal/common/ucx: add periodical flush and counter to opal directory.
Signed-off-by: Xin Zhao <xinz@mellanox.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
1fa7054041 opal/common/ucx: use trylock in opal_common_progress
Signed-off-by: Xin Zhao <xinz@mellanox.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
2d3cffe1a3 opal/common/ucx: replace opal_mutex_t with opal_recursive_mutex_t
Signed-off-by: Xin Zhao <xinz@mellanox.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
aa26a724ed opal/common/ucx: introduce internal UCX request in wpool.
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
07cb4134be opal/common/ucx: Set of bug fixes in wpool
Signed-off-by: Xin Zhao <xinz@mellanox.com>
2019-02-19 14:22:07 -08:00
Xin Zhao
344bb641a1 opal/common/ucx: Minor changes in wpool
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
9fb9cfbe8e opal/common/ucx: Simplify Worker Pool TLS structure
Get rid of unneeded context and memory region identifiers

Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:07 -08:00
Artem Polyakov
1e7bf7085d opal/common/ucx: Improve/fix debug output macro's
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Artem Polyakov
fd98ee14eb opal/common/ucx: Code cleanup
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Artem Polyakov
f38c9f3e5f opal/common/ucx: Simplify Worker Pool memory handler
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Artem Polyakov
6b7acdf21f opal/common/ucx: Somplify Worker Pool context management
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Xin Zhao
8b7fa927ba opal/common/ucx: Add fetch primitives to wpool
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Xin Zhao
bfbf818fe1 opal/common/ucx: Complete initialization of the Worker Pool
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Artem Polyakov
e28fadb048 opal/common/ucx: Introduce Worker Pool (wpool) functionality
Worker Pool is an object containing/managing a set of UCX workers
and providing access to those workers through a smal interface
to allow Multi-Threaded applicatoins to access multiple HW contexts.

Signed-off-by: Artem Polyakov <artpol84@gmail.com>
2019-02-19 14:22:06 -08:00
Jeff Squyres
8b60cc8039 opal/util: remove opal_strncpy()
This function is not used anywhere anymore.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-13 10:21:39 -08:00
Ralph Castain
24a64524ae Correctly name the package so the flags get set
Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-08 15:11:48 -08:00
Ralph Castain
fc0b0938a7 Cache the old orte/regx components
In case someone wants to restore and use them - leave them as
opal_ignore'd for now

Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-08 11:25:35 -08:00
Gilles Gouaillardet
88ac05fca6 misc fixes
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2019-02-08 11:11:25 -08:00
Ralph Castain
125d236173 Move from the use of regex to compression
We've been fighting the battle of trying to create a regex generator and
parser that can handle arbitrary hostname schemes - without long-term
success. The worst of it is that there is no way of checking to see if
the computed regex is correct short of parsing it and doing a
character-by-character comparison with the original string. Ugh...there
has to be a better solution.

One option is to investigate using 3rd-party regex libraries as
those are coming from communities whose sole focus is resolving that
problem. However, someone would need to spend the time to investigate
it, and we'd have to find a license-friendly implementation.

Another option is to quit beating our heads against the wall and just
compress the information. It won't be as much of a reduction, but we
also won't keep hitting scenarios where things break. In this case, it
seems that "perfection" is definitely the enemy of "good enough".

This PR implements the compression option while retaining the
possibility of people adding regex-generating components. The
compression code used in ORTE is consolidated into the opal/compress
framework. That framework currently held bzip and gzip components for
use in compressing checkpoint files - since we no longer support C/R, I
have .opal_ignore'd those components.

However, I have left the original framework APIs alone in case someone
ever decides to redo C/R. The APIs of interest here are added to the
framework - specifically, the "compress_block" and "decompress_block"
functions. I then moved the ORTE zlib compression code into a new
component in this framework.

Unfortunately, the framework currently is a single-select one - i.e.,
only one active component at a time. Since I .opal_ignore'd the other
two and made the priority of zlib high, this isn't a problem. However,
if someone wants to re-enable bzip/gzip or add another component, they
might need to transition opal/compress to a multi-select framework.

Included changes:

* Consolidate the compression code into the opal/compress framework

* Move the ORTE zlib compression code into a new opal/compress/zlib
  component

* Ignore the bzip and gzip components in opal/compress framework

* Add a "compress_base_limit" MCA param to set the threshold above which
  we compress data - defaults to 4096 bytes

* Delete stale brucks and rcd components from orte/grpcomm framework

* Delete the orte/regx framework

* Update the launch system to use opal/compress instead of string regex

* Provide a default module if no zlib is available

* Fix some misc multi-node issues

* Properly generate the nidmap in response to a "connection warmup"
  message so the remote daemon knows the children it needs to launch.

* Remove stale references to orte_node_regex

* opal_byte_object_t's are not OPAL objects - properly release allocated
  memory.

* Set the topology

* Currently only handling homogeneous case

* Update the compress framework files to conform

* Consolidate open/close into one "frame" file. Ensure we open/close the
  framework

Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-08 11:11:14 -08:00
KAWASHIMA Takahiro
8bbd201029
Merge pull request #6205 from kawashima-fj/pr/fp16
Add FP16 datatypes
2019-02-08 14:52:13 +09:00
Jeff Squyres
dd20174532 Remove opal/mca/common/ofi.
It never lived up to its purpose (and has caused amorphous indirect
errors such as https://github.com/open-mpi/ompi/issues/2519), so
delete it.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 06:29:58 -08:00
Jeff Squyres
f5e1a672cc ofi: revamp OPAL_CHECK_OFI configury
Update the OPAL_CHECK_OFI configury macro:

- Make it safe to call the macro multiple times:
  - The checks only execute the first time it is invoked
  - Subsequent invocations, it just emits a friendly "checking..."
    message so that configure output is sensible/logical
- With the goal of ultimately removing opal/mca/common/ofi, rename the
  output variables from OPAL_CHECK_OFI to be
  opal_ofi_{happy|CPPFLAGS|LDFLAGS|LIBS}.
- Update btl/ofi, btl/usnic, and mtl/ofi for these new conventions.
- Also, don't use AC_REQUIRE to invoke OPAL_CHECK_OFI because that
  causes the macro to be invoked at a fairly random time, which makes
  configure stdout confusing / hard to grok.
- Remove a little left-over kruft in OPAL_CHECK_OFI, too (which
  resulted in an indenting change, making the change to
  opal_check_ofi.m4 look larger than it really is).

Thanks Alastair McKinstry for the report and initial fix.
Thanks Rashika Kheria for the reminder.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 06:29:58 -08:00
Jeff Squyres
b556cabfe9 btl/ofi/Makefile.am: down with tabs!
Replace all tabs with spaces.  No code or logic changes.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 06:29:58 -08:00
Jeff Squyres
59c8ab6da4 m4: remove all configury related to libibverbs
Now that all components that use libibverbs are gone, remove
OPAL_CHECK_VERBS and the confusingly-named OPAL_CHECK_OPENFABRICS
(which really just checked for verbs things -- not all the possible
OpenFabrics APIs/libraries).

The only code left in Open MPI that calls verbs is hwloc -- and that's
just the APIs that takes an IBV device and returns topological
information about it.  Since nothing in the Open MPI code base uses
the "ibv_*" API any more, we have no need for this hwloc functionality
so we'll even remove the --with-verbs configure options.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 05:36:06 -08:00
Jeff Squyres
3f4af8e51c opal/common: remove stale common components
The verbs and verbs_usnic components are now no longer necessary / no
longer used anywhere in the code base.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 05:36:06 -08:00
Jeff Squyres
8de786f5a4 btl/openib: So long / farewell / it's time to say goodnight
So long BTL openib!  After many years of (mostly) faithful service, it
is time to remove the openib BTL.  It has been fully replaced by other
components, such as the UCX PML and OFI MTL.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-02-07 05:34:19 -08:00
Ralph Castain
677ce0a69f Update PMIx configure logic in the embedded component
PMIx is removing the --enable-embedded-libevent and
--enable-embedded-hwloc flags as they are confusing users. Instead, we
will use the --enable-embedded-mode to handle both of these options.
Update the embedded configury to handle it.

Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-06 17:15:44 -08:00
Matias Cabral
5aef3148d3
Merge pull request #6351 from aravindksg/fix_btl_ofi_valgrind
btl/ofi: Fix valgrind complaints on uninitialized pointer use
2019-02-05 16:36:45 -08:00
Ralph Castain
baef25338a Update to latest PRI master
Signed-off-by: Ralph Castain <rhc@pmix.org>
2019-02-04 10:10:58 -08:00
Aravind Gopalakrishnan
786e686d43 btl/ofi: Fix valgrind complaints on uninitialized pointer use
It doesn't seem like the BTL was using uninitialized pointer. But simply
setting the rcache pointer to NULL after destroying it makes the valgrind
errors go away.

Fixes Issue #6345

Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
2019-02-01 14:03:23 -08:00