1
1
Граф коммитов

10893 Коммитов

Автор SHA1 Сообщение Дата
Gilles Gouaillardet
26e42f9a0c op/avx: check for _mm512_mullo_epi64() AVX512 intrinsic
PGI (20.4) compiler do not define this intrinsic, so only build
AVX512 support if _mm512_mullo_epi64() intrisic is defined.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2020-11-04 14:45:03 +09:00
Yossi Itigin
1f3e33441c
Merge pull request #8140 from hoopoepg/topic/pml-ucx-recv-improved-errhandling
PML/UCX: improved error processing in MPI_Recv
2020-11-03 13:08:42 +02:00
Jeff Squyres
e9e5dab8b9
Merge pull request #8153 from dancejic/multi
Using package_rank to select between NIC of equal distance from the process.
2020-11-02 15:27:37 -05:00
Sergey Oblomov
eb9405d53f PML/UCX: improved error processing in MPI_Recv
- improved error processing in MPI_Recv implementation
  of pml UCX
- added error handling for pml_ucx_mrecv call

Signed-off-by: Sergey Oblomov <sergeyo@nvidia.com>
2020-11-02 11:25:28 +02:00
Nikola Dancejic
8017f12801 Using package_rank to select between NIC of equal distance from the process.
If PMIX_PACKAGE_RANK is available, uses this value to select between multiple
NIC of equal distance between the current process. If this value is not
available, try to calculate it by getting the locality string from each local
process and assign a package_rank. If everything fails, fall back to using
process_id.rank to select the NIC. This last case is not ideal, but has a small
chance of occuring, and causes an output to be displayed to notify that this is
occuring.

Signed-off-by: Nikola Dancejic <dancejic@amazon.com>
2020-11-02 00:32:03 -08:00
Fangcong-Yin
7ee34e9c20 Convert 12 .3in files to md
Signed-off-by: Fangcong Yin <fyin2@nd.edu>
Convert MPI_Graph_create.3in - MPI_Group_intersection.3in to md files

Update ompi/mpi/man/man3/MPI_Grequest_complete.md

Signed-off-by: Fangcong Yin <fyin2@nd.edu>

Co-authored-by: Jeff Squyres <jsquyres@users.noreply.github.com>

Update ompi/mpi/man/man3/MPI_Group_excl.md

Signed-off-by: Fangcong Yin <fyin2@nd.edu>

Co-authored-by: Jeff Squyres <jsquyres@users.noreply.github.com>
2020-10-31 15:09:26 -04:00
Jeff Squyres
6f2a5b91cd
Merge pull request #8162 from Colton-K/pr/MPI_C_batch2
Converted batch 2 of MPI_C* (MPI_Comm_accept - MPI_Comm_dup_with_info)
2020-10-31 13:20:14 -04:00
Colton Kammes
ee3bd5859e Converted batch 2 of MPI_C* (MPI_Comm_accept - MPI_Comm_dup_with_info)
Signed-off-by: Colton Kammes <ckammes@nd.edu>
2020-10-31 10:02:37 -04:00
Jeff Squyres
c7b968ef20
Merge pull request #6903 from ggouaillardet/topic/use-mpi-f08-profile
fortran/use-mpi-f08: generates PMPI bindings from the MPI bindings
2020-10-31 09:41:49 -04:00
Jeff Squyres
400c005d04
Merge pull request #8161 from Fangcong-Yin/latest_pr
Convert MPI_Gather.3in - MPI_Get_version.3in to md
2020-10-30 21:13:54 -04:00
Fangcong-Yin
d85bf3ae1a Convert MPI_Gather.3in - MPI_Get_version.3in to md
Signed-off-by: Fangcong Yin (fyin2@nd.edu)
2020-10-30 20:03:59 -04:00
Colton Kammes
a08f8d30dd Converted first half of MPI_C* (from MPI_Cancel to MPI_Close_port)
Signed-off-by: Colton Kammes <ckammes@nd.edu>
2020-10-30 18:25:22 -04:00
Jeff Squyres
25e47411e0
Merge pull request #8159 from jsquyres/pr/fix-coll-and-adapt-warnings
coll/adapt and coll/han: fix trivial compiler warnings
2020-10-30 14:34:16 -04:00
Jeff Squyres
f813656d24
Merge pull request #8154 from Fangcong-Yin/latest_pr
Convert MPI_File_write_ordered.3in - MPI_Free_mem.3in to md
2020-10-30 10:44:23 -04:00
Jeff Squyres
ee405ccaa5 coll/adapt and coll/han: fix trivial compiler warnings
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2020-10-30 10:41:14 -04:00
Jeff Squyres
548f5dbe47
Merge pull request #8135 from Colton-K/pr/all-MPI_B
Convert rest of the MPI_B* man pages to Markdown
2020-10-30 10:30:51 -04:00
Fangcong-Yin
82081118d6 Convert MPI_File_write_ordered.3in - MPI_Free_mem.3in to md
Signed-off-by: Fangcong Yin (fyin2@nd.edu)

Convert MPI_File_write_ordered.3in - MPI_Free_mem.3in to md
Signed-off-by: Fangcong Yin (fyin2@nd.edu)

Convert MPI_File_write_ordered.3in - MPI_Free_mem.3in to md
Signed-off_by: Fangcong Yin (fyin2@nd.edu)
2020-10-30 10:11:19 -04:00
Colton Kammes
cded23fe9b Finished converting MPI_B section into markdown files
Signed-off-by: Colton Kammes <ckammes@nd.edu>
2020-10-30 09:19:01 -04:00
Gilles Gouaillardet
8c2b916048 fortran/use-mpi-tkr: revamp PMPI_* declarations
only declare MPI_* bindings and use macros to deduce the PMPI_* bindings

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2020-10-30 14:00:42 +09:00
Gilles Gouaillardet
e981e46446 fortran/use-mpi-ignore-tkr: revamp PMPI_* declarations
only declare MPI_* bindings and use macros to deduce the PMPI_* bindings

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2020-10-30 14:00:42 +09:00
Gilles Gouaillardet
c2e3df6d3e fortran/use-mpi-f08: simplify mod/mpi-f08-interfaces.h
have this file generated at configure time and use the
@OMPI_FORTRAN_IGNORE_TKR_PREDECL@ and @OMPI_FORTRAN_IGNORE_TKR_TYPE@
variable for buffer definitions

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2020-10-30 13:58:48 +09:00
Gilles Gouaillardet
5219241d13 fortran/use-mpi-f08: generates PMPI bindings from the MPI bindings
- via the OMPI_BUILD_MPI_PROFILING macro, avoid the need to have two distinct
   source files for MPI_* and PMPI_* bindings.
 - move the PMPI bindings into ompi/mpi/fortran/use-mpi-f08/profile.
 - remove an useless dependency to mpi-f08.lo
 - share most of mod/[p]mpi-f08-interfaces.F90 code in a single mod/mpi-f08-interfaces.h

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2020-10-30 13:58:48 +09:00
bosilca
d330551547
Merge pull request #8146 from bosilca/fix/han_coverity
Fix HAN issues reported by Coverity.
2020-10-28 18:40:14 -04:00
George Bosilca
154117515a Fix HAN issues reported by Coverity.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2020-10-28 15:44:16 -04:00
Jeff Squyres
00ccc9d936
Merge pull request #8136 from bmwiedemann/date
configury: make build reproducible
2020-10-28 07:42:31 -04:00
Gilles Gouaillardet
7b4e8ba4aa configury: make build Reproducible
If defined, use SOURCE_DATE_EPOCH environment variable; make the build
Reproducible by forcing timestamps.  See
https://reproducible-builds.org/docs/source-date-epoch/ for more
information.

Thanks Bernhard M. Wiedemann for bringing this to our attention.

Fixes open-mpi/ompi#3759

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
Signed-off-by: Bernhard M. Wiedemann <bwiedemann@suse.de>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2020-10-28 03:15:38 +01:00
Jeff Squyres
593a918379
Merge pull request #8133 from Fangcong-Yin/latest_pr
Convert MPI_Bcast.3in to markdown
2020-10-27 10:22:58 -04:00
Fangcong-Yin
c2ef4af7b6 Convert MPI_Bcast.3in to markdown
Signed-off-by: Fangcong Yin (fyin2@nd.edu)
2020-10-26 18:53:00 -04:00
Howard Pritchard
487bbf31ba
Merge pull request #8052 from hkuno/john.l.byrne/ofi_create_recv_tag_mask
use sync_send mask for ofi_create_recv_tag
2020-10-26 15:56:35 -06:00
Raghu Raja
2e5303f618
Merge pull request #8124 from rajachan/peek-probe-errorflow
mtl/ofi: Fix erroneous FI_PEEK/FI_CLAIM usage
2020-10-26 10:42:12 -07:00
bosilca
ce97090673
Merge pull request #7735 from bosilca/coll/han
A hierarchical, architecture-aware collective communication module
2020-10-26 00:07:03 -04:00
George Bosilca
16b49dc5b3 A complete overhaul of the HAN code.
Among many other things:
- Fix an imbalance bug in MPI_allgather
- Accept more human readable configuration files. We can now specify
  the collective by name instead of a magic number, and the component
  we want to use also by name.
- Add the capability to have optional arguments in the collective
  communication configuration file. Right now the capability exists
  for segment lengths, but is yet to be connected with the algorithms.
- Redo the initialization of all HAN collectives.

Cleanup the fallback collective support.
- In case the module is unable to deliver the expected result, it will fallback
  executing the collective operation on another collective component. This change
  make the support for this fallback simpler to use.
- Implement a fallback allowing a HAN module to remove itself as
  potential active collective module, and instead fallback to the
  next module in line.
- Completely disable the HAN modules on error. From the moment an error is
  encountered they remove themselves from the communicator, and in case some
  other modules calls them simply behave as a pass-through.

Communicator: provide ompi_comm_split_with_info to split and provide info at the same time
Add ompi_comm_coll_preference info key to control collective component selection

COLL HAN: use info keys instead of component-level variable to communicate topology level between abstraction layers
- The info value is a comma-separated list of entries, which are chosen with
  decreasing priorities. This overrides the priority of the component,
  unless the component has disqualified itself.
  An entry prefixed with ^ starts the ignore-list. Any entry following this
  character will be ingnored during the collective component selection for the
  communicator.
  Example: "sm,libnbc,^han,adapt" gives sm the highest preference, followed
  by libnbc. The components han and adapt are ignored in the selection process.
- Allocate a temporary buffer for all lower-level leaders (length 2 segments)
- Fix the handling of MPI_IN_PLACE for gather and scatter.

COLL HAN: Fix topology handling
 - HAN should not rely on node names to determine the ordering of ranks.
   Instead, use the node leaders as identifiers and short-cut if the
   node-leaders agree that ranks are consecutive. Also, error out if
   the rank distribution is imbalanced for now.

Signed-off-by: Xi Luo <xluo12@vols.utk.edu>
Signed-off-by: Joseph Schuchart <schuchart@icl.utk.edu>
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2020-10-25 18:13:16 -04:00
Jeff Squyres
6304c3f57c
Merge pull request #8131 from jsquyres/pr/include-mpi-man5-markdown-nroffed-pages
ompi/man5: include nroff-ified pages in tarball
2020-10-25 17:49:22 -04:00
Colton Kammes
f14a6c5089 Converted MPI_Barrier.3in to MPI_Barrier.md
Signed-off-by: Colton Kammes <ckammes@nd.edu>
2020-10-25 14:12:31 -04:00
Jeff Squyres
ca524134d4 ompi/man5: include nroff-ified pages in tarball
Add a line that was accidentally left out that includes the generated
nroff pages (from the Markdown pages) in the dist tarball.

The lack of these files in the dist tarball was causing configure to
fail a test and therefore conclude that we need to have pandoc
installed.  Put differently: it's a specific goal that we do not want
end users to have to have Pandoc installed -- all Markdown files must
generate their nroff equivalents and have those nroff files included
in the distribution tarball.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2020-10-24 10:39:50 -07:00
Raghu Raja
39f8a86b65 mtl/ofi: Fix erroneous FI_PEEK/FI_CLAIM usage
The current iprobe/improbe implementations merely checks the return
code on the posted receive operation to tell if there is a match or
not. This commit moves the check to the probe's error callback
instead. Per the semantics defined in libfabric, the peek operation is
asynchronous and the results are to be fetched from the completion
queue. If no message is found matching the tags specified in the peek
request, then a completion queue error entry with err field set to
FI_ENOMSG will be available.

Signed-off-by: Raghu Raja <craghun@amazon.com>
2020-10-23 17:07:16 +00:00
Raghu Raja
415dddb9af mtl/ofi: Do not fail if error CQ is empty
In multi-threaded scenarios, any thread that attempts to read a CQ
when there's a pending error CQ entry gets an -FI_EAVAIL. Without
any serialization here (which is okay, since libfabric will protect
access to critical CQ objects), all threads proceed to read from the
error CQ, but only one thread fetches the entry while others get
-FI_EAGAIN indicating an empty queue, which is not erroneous.

Signed-off-by: Raghu Raja <craghun@amazon.com>
2020-10-21 21:27:29 +00:00
Nysal Jan K A
992e8f91bb
Merge pull request #8105 from AboorvaDevarajan/fix_zero2
pml/ucx: fix zero sized datatype transfers
2020-10-20 19:53:15 +05:30
Austen Lauria
23269ccaa6
Merge pull request #8079 from markalle/exported_symbol_names
symbol pollution
2020-10-13 11:48:43 -04:00
Ralph Castain
92d28420d1
Protect against non-standard environment
Environments are supposed to provide the PMIX_NODE_RANK - if they don't,
we can probably safely assume the same as local rank. Protect against a
few other values that might not be provided.

Signed-off-by: Ralph Castain <rhc@pmix.org>
2020-10-13 07:14:20 -07:00
Aboorva Devarajan
202b81d95c pml/ucx: fix zero sized datatype transfers
Signed-off-by: Aboorva Devarajan <aburvadevarajan@gmail.com>
2020-10-13 01:58:50 -04:00
Ralph Castain
9db78938c4
Add some protection against PMIx v4
Ensure we can build against PMIx v3.1.5

Fixes #8089

Signed-off-by: Ralph Castain <rhc@pmix.org>
2020-10-12 13:54:47 -07:00
bsergentm
220b997a58 Coll/han Bull
* first import of Bull specific modifications to HAN

* Cleaning, renaming and compilation fixing Changed all future into han.

* Import BULL specific modifications in coll/tuned and coll/base

* Fixed compilation issues in Han

* Changed han_output to directly point to coll framework output.

* The verbosity MCA parameter was removed as a duplicated of coll verbosity

* Add fallback in han reduce when op cannot commute and ppn are imbalanced

* Added fallback wfor han bcast when nodes do not have the same number of process

* Add fallback in han scatter when ppn are imbalanced

+ fixed missing scatter_fn pointer in the module interface

Signed-off-by: Brelle Emmanuel <emmanuel.brelle@atos.net>
Co-authored-by: a700850 <pierre.lemarinier@atos.net>
Co-authored-by: germainf <florent.germain@atos.net>
2020-10-09 14:17:46 -04:00
Xi Luo
182c333b21 Initial import of the HAN collective module
a hierarchical, architecture-aware collective communication module.

Add Reduce and remove up_seg_size and low_seg_size in Bcast
Increase HAN's priority

Signed-off-by: Xi Luo <xluo12@vols.utk.edu>
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2020-10-09 14:17:46 -04:00
bosilca
a541ab933d
Merge pull request #8066 from devreal/spc-pml-ucx
PML UCX: add SPC instrumentation for sent/received message sizes
2020-10-07 21:56:27 -04:00
Mark Allen
34b42b1b09 symbol pollution
Made a couple vars static if they didn't look like they were used
more than one place, and added prefixes to a few.

Signed-off-by: Mark Allen <markalle@us.ibm.com>
2020-10-07 13:04:39 -04:00
bosilca
9f114c6afe
Merge pull request #8067 from devreal/fix-ob1-spc
PML OB1: Fix potential overcounting of SPC sent/received message sizes and account for fin messages
2020-10-05 22:26:24 -04:00
Brian Barrett
0e9581d478 build: Move hwloc to a 3rd-party package
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.

This patch moves the hwloc library bundled with Open MPI from a
MCA framework to a stand-alone library built outside of OPAL.  Due
to the amount of code in the MCA base (and its assumptions about
being part of an MCA framework), the framework is left with no
active components.  Any pre-installed version of HWLOC 1.6 or
newer is preferred over the internal version.

Signed-off-by: Brian Barrett <bbarrett@amazon.com>
2020-10-01 16:55:59 +00:00
Brian Barrett
9ffac85650 build: Move libevent to a 3rd-party package
With Open MPI 5.0, the decision was made to stop building
3rd-party packages, such as Libevent, HWLOC, PMIx, and PRRTE as
MCA components and instead 1) start relying on external libraries
whenever possible and 2) Open MPI builds the 3rd party
libraries (if needed) as independent libraries, rather than
linked into libopen-pal.

This patch moves libevent from an MCA framework to a stand-alone
library built outside of OPAL.  A wrapper in opal/util is provided
to minimize the unnecessary changes in the rest of the code.  When
using the internal Libevent, it will be installed as a stand-alone
libevent.a, instead of bundled in OPAL.  Any pre-installed version
of Libevent at or after 2.0.21 is preferred over the internal
version.

Signed-off-by: Brian Barrett <bbarrett@amazon.com>
2020-10-01 16:55:58 +00:00
Nathan Hjelm
1f5ed0b83d
Merge pull request #8070 from devreal/osc-page-align
OSC RDMA: put memory for each process into separate pages
2020-09-29 07:54:45 -06:00