1
1
Граф коммитов

9698 Коммитов

Автор SHA1 Сообщение Дата
George Bosilca
409638bdf4 Keep the out-of-sequence fragment ordered.
Rework the logic to handle the out-of-sequence fragments on the receiver
side. A large number of OOS messages are still arriving even in single
threaded scenarios.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-11-08 14:27:13 -05:00
Thananon Patinyasakdikul
e3b267a8fe pml/ob1: match callback will now queue wrong sequence frag and return.
In multithreaded case, it is expensive to release the lock, call the slow match
and retake the lock again just to queue the frag. This patch will eliminate number of
lock taken by queueing the frag right away and return.

Signed-off-by: Thananon Patinyasakdikul <tpatinya@utk.edu>
2017-10-30 12:01:50 -04:00
Edgar Gabriel
4c0d347412 Merge pull request #4230 from edgargabriel/topic/no-smart-fview
io/ompio: add a new grouping option avoiding communication
2017-09-26 10:56:06 -05:00
bosilca
f44e674992 Merge pull request #4074 from bosilca/topic/coverity
Fix coverity complaints.
2017-09-25 15:05:31 -04:00
Guillaume Mercier
4e7c130c31
Add correct reordering computation in partially distributed case.
Replaced matching array with k and bcast with scatter.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
Signed-off-by: Guillaume Mercier <mercier@labri.fr>
2017-09-25 13:10:11 -04:00
George Bosilca
3dd1d8cb53
Delay the first check for the HWLOC topology.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-25 13:09:57 -04:00
George Bosilca
28046b37df
Always succesfully return.
As the reordering is an optional step, if any operation during the
reorder fails we can return the duplicata of the original communicator
associated with the topology information.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-25 12:53:20 -04:00
George Bosilca
219a96fa69
Prevent memory leaks.
Reorder the code to simplify the memory management.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-25 12:53:20 -04:00
George Bosilca
64bff0e326
Disable monitoring if we compile statically.
Protect all components against compilation on static builds.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-25 12:18:23 -04:00
George Bosilca
458ccc12e1
Move the profiling library in common/monitoring
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-25 12:18:23 -04:00
Clément FOYER
f334607c34
Simplify the communicator's name caching management (#6)
Signed-off-by: Clement Foyer <clement.foyer@inria.fr>
2017-09-25 12:18:23 -04:00
bosilca
a680b3ac6d Merge pull request #3853 from clementFoyer/master
OMPI monitoring: Simplify the communicator's name caching management + misc test changes
2017-09-25 12:14:36 -04:00
yohann
1f8cabc890 mtl/ofi: Fix provider selection.
This allows mtl_ofi_provider_include to work with layered providers as well.
e.g. --mca mtl_ofi_provider_include "providerX;ofi_rxm"

Signed-off-by: yohann <yohann.burette@intel.com>
2017-09-20 16:00:50 -07:00
Gilles Gouaillardet
b9315edb85 configury: remove the --disable-mpi-io option
Fixes open-mpi/ompi#2185

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-09-20 14:39:09 +09:00
Edgar Gabriel
76a8c67575 io/ompio: add a new grouping option avoiding communication
the new grouping option simple+ performs all calculations used
for the aggregator selection as if the default file view would be used,
thus avoiding communication in file_set_view all together. This mode
is useful for applications that do not set a file view, but use
explicit offset operations on the default file view.

Signed-off-by: Edgar Gabriel <egabriel@central.uh.edu>
2017-09-18 12:30:34 -05:00
Ralph Castain
ed508010b4 Remove stale tools
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-18 07:30:47 -07:00
Ralph Castain
3c914a7a97 Complete the fix of the ORTE DVM. We will now use "prun" instead of "orterun -hnp foo" to execute jobs. This provides the feature of automatic discovery of the orte-dvm so you don't need to manually enter URI's or contact file locations. All IO is forwarded to prun.
Still in the "needs to be done" category:

* mapping/ranking/binding options aren't correctly supported

* if the DVM encounters some errors (e.g., not enough resources for the job), the resulting error is globally set and impacts any subsequent job submission

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-16 13:13:07 -07:00
Ralph Castain
3f8908871b Since the DVM is now tied to prun, don't build the DVM either unless prun can be built
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-13 11:55:10 -07:00
Brian Barrett
637ebf60f9 atomics: Remove requirement of 64 bit atomics
Remove two of the three  instances of components requiring
64 bit atomics, even on 32 bit systems.  The SM OSC component
also uses 64 bit atomics, but is a more complicated fix that
will follow this one.  Currently, no one is testing on
platforms that don't provide 64 bit atomics (even in 32 bit
mode), but with the removal of the non-inline assembly for
IA32, the older compilers on Absoft's test systems now
result in no practical way to call cmpxchg8 in 32 bit mode.
At that point, these failures started popping up.

Signed-off-by: Brian Barrett <bbarrett@amazon.com>
2017-09-11 19:50:10 -07:00
Nathan Hjelm
7cdda24206 osc/sm: do not require 64-bit atomic math
This commit fixes a compile issue on 32-bit systems that do not
support 64-bit atomic math. The active target path was using 64-bit
atomics exclusively to support PSCW. This commit updates the code to
use either 32 or 64-bit atomic math depending on what is available.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2017-09-11 14:10:38 -10:00
Nathan Hjelm
4bba8774f4 monitoring: fix MPI_T regression
The monitoring code causes MPI_T based tools to segfault when
monitoring is disabled. This happens because the performance
variables remain registered after the common/monitoring
component is dlclosed due to a missing variable registration
flag. This commit adds the necessary flag to all the registered
performance variables.

The issue on github is #4162. Close when applied to master.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2017-09-06 14:24:35 -06:00
bosilca
dc538e9675 Merge pull request #1177 from bosilca/topic/large_msg
Topic/large msg
2017-09-05 13:30:19 -04:00
Gilles Gouaillardet
ecb6b81a05 mpi: correctly handle MPI_IN_PLACE by memchecker in neighborhood collectives
MPI_IN_PLACE is not a valid send buffer for neighborhood collectives, so do not
invoke memchecker in this case.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-09-04 11:21:32 +09:00
Gilles Gouaillardet
66c9485e77 MPI_Isend: memchecker do not mark send buffer as unaccessible after pml isend invokation
Today's MPI standard mandates the send buffer remains accessible during the send operation.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-09-04 11:21:32 +09:00
Gilles Gouaillardet
af8242a121 pml/ob1: have memchecker make recv buffer defined again when mca_pml_ob1_recv completes
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-09-04 11:18:05 +09:00
Gilles Gouaillardet
6ee9366243 MPI_Wait: correctly handle MPI_STATUS_IGNORE in MEMCHECKER
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-09-04 11:18:05 +09:00
Aravind Gopalakrishnan
2e83cf15ce Add support for GPU buffers for PSM2 MTL
PSM2 enables support for GPU buffers and CUDA managed memory and it can
directly recognize GPU buffers, handle copies between HFIs and GPUs.
Therefore, it is not required for OMPI to handle GPU buffers for pt2pt cases.
In this patch, we allow the PSM2 MTL to specify when
it does not require CUDA convertor support. This allows us to skip CUDA
convertor init phases and lets PSM2 handle the memory transfers.

This translates to improvements in latency.
The patch enables blocking collectives and workloads with GPU contiguous,
GPU non-contiguous memory.

Signed-off-by: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@intel.com>
2017-09-01 16:59:03 -07:00
George Bosilca
866899e836
Always abide to the RDMA pipeline limit.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-01 18:52:48 -04:00
George Bosilca
050bd3b6d7
Make the pipeline depth an int instead of a size_t. While
they are supposed to be unsigned, casting them to a signed
value for all atomic operations is as errorprone as handling
them as signed entities.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-09-01 18:52:48 -04:00
Clement Foyer
9a8fc1b9f1 Simplify the communicator's name caching management
Remove useless over-initialization

Signed-off-by: Clement Foyer <clement.foyer@inria.fr>
2017-08-29 12:52:47 +02:00
Yossi Itigin
14a93a5992 pml_ucx: fix tag/context_id layout and upper bounds.
Signed-off-by: Yossi Itigin <yosefe@mellanox.com>
2017-08-27 17:15:48 +03:00
Josh Hursey
ad87aa2674 Merge pull request #4121 from jjhursey/explore/dlopen-local
mca: Dynamic components link against project lib
2017-08-25 13:15:51 -05:00
Joshua Hursey
49c40f05d4 mpi/java: Remove dlopen() workaround
* See discussion on Issue #3705 regarding why this is no longer needed.

Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
2017-08-24 11:56:17 -04:00
Joshua Hursey
e1d079544b mca: Dynamic components link against project lib
* Resolves #3705
 * Components should link against the project level library to better
   support `dlopen` with `RTLD_LOCAL`.
 * Extend the `mca_FRAMEWORK_COMPONENT_la_LIBADD` in the `Makefile.am`
   with the appropriate project level library:
```
MCA components in ompi/
       $(top_builddir)/ompi/lib@OMPI_LIBMPI_NAME@.la
MCA components in orte/
       $(top_builddir)/orte/lib@ORTE_LIB_PREFIX@open-rte.la
MCA components in opal/
       $(top_builddir)/opal/lib@OPAL_LIB_PREFIX@open-pal.la
MCA components in oshmem/
       $(top_builddir)/oshmem/liboshmem.la"
```

Note: The changes in this commit were automated by the script in
the commit that proceeds it with the `libadd_mca_comp_update.py`
script. Some components were not included in this change because
they are statically built only.

Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
2017-08-24 11:56:16 -04:00
Ralph Castain
e02c39385a Merge branch 'master' into topic/modex 2017-08-22 20:06:35 -07:00
George Bosilca
50f471e31e
Cleanup a set of warnings reported by Ralph.
Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
2017-08-22 23:00:18 -04:00
Ralph Castain
d80b0c7990 If the HWLOC shared memory system is unable to connect, then fallback to providing the topology via XML. Do not automatically provide the XML to every process as that defeats the purpose of the shared memory system. Instead, use PMIx_Query_info_nb to get the info from the server when required.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-08-22 18:12:26 -07:00
Jeff Squyres
ea5093fc14 mpi/info_delete: fix return code
Per MPI-3.1, ensure to raise an MPI exception with value
MPI_ERR_INFO_NOKEY if we try to MPI_INFO_DELETE a key that does not
exist.  Thanks to @dalcinl (Lisando Dalcin) for raising the issue.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2017-08-22 08:56:40 -07:00
Gilles Gouaillardet
a3e31fa8d0 ompi/communicator: plug a memory leak in ompi_comm_init()
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-08-21 11:47:11 +09:00
Edgar Gabriel
99c7482dd8 Merge pull request #3739 from cniethammer/sharedfp_sm_file_dir
Create file for file backed shared memory in process job session dir.
2017-08-15 11:53:30 -05:00
Edgar Gabriel
8fe1c63e25 io/ompio: change the increment for cost based aggr. selection
- change the increment used to test various no. of aggregators
  to avoid using only power of two numbers
- convert some paratemers in the cost function from integers to
  to floats for providing smoother and more consistent results
- set the FVIEW_IS_SET flag on the file *only* if the user
  has set anything else than the default file view.

Signed-off-by: Edgar Gabriel <gabriel@cs.uh.edu>
2017-08-15 09:50:41 -05:00
Edgar Gabriel
f258036e06 fcoll/two_phase: adjust aggregator selection to new mapby flag on MPI_COMM_WORLD
adjust how the aggregator nodes are selected depending on whether processes
have been mapped by node or anything else.

Signed-off-by: Edgar Gabriel <gabriel@cs.uh.edu>
2017-08-15 09:50:41 -05:00
Edgar Gabriel
92eff9050c communicator/comm_init.c: add a new flag indicating binding policy
Check for the binding policy used. We are only interested in
whether mapby-node has been set right now (could be extended later)
and only on MPI_COMM_WORLD, since for all other sub-communicators
it is virtually impossible to identify their layout across nodes
in the most generic sense. This is used by OMPIO for deciding which
ranks to use for aggregators

Signed-off-by: Edgar Gabriel <gabriel@cs.uh.edu>
2017-08-15 09:50:41 -05:00
Edgar Gabriel
b3f59c76e1 io/ompio: new simple aggr. selection algorithm
add a new aggregator selection algorithm based on the performance
model described in:

Shweta Jha, Edgar Gabriel,
'Performance Models for Communication in Collective I/O Operations'
Proceedings of the 17th IEEE/ACM Symposium
on Cluster, Cloud and Grid Computing, Workshop on Theoretical
Approaches to Performance Evaluation, Modeling and Simulation, 2017.

Signed-off-by: Edgar Gabriel <gabriel@cs.uh.edu>
2017-08-15 09:50:41 -05:00
Jeff Squyres
791bcee6c0 ompi/fortran: remove proof-of-concept mpi_f08 module
This module was always intended to be a proof of concept, and was far
from complete.  If/when someone implemented F08 descriptor support for
the mpi_f08 module, this commit can either be restored or used as
reference material.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2017-08-10 06:19:17 -07:00
Gilles Gouaillardet
dfe7b2be3f fortran/use-mpi-f08-desc: add a missing include file
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-08-09 13:19:22 +09:00
Gilles Gouaillardet
2c71c27882 fortran2008: fix mpiext example
in order to solve an egg and the chicken problem,  in which mpiext need mpi-f08-types.mod
and/but use-mpi-f08[-desc] needs mpiext, add an extra step
- build fortran 2008 modules only
- build fortran 2008 mpi extensions
- and then build fortran 2008 bindings

Fixes open-mpi/ompi#3605

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-08-09 13:19:22 +09:00
bosilca
9b43de112c Merge pull request #4014 from bosilca/topic/treematch
Topic/treematch
2017-08-08 11:28:22 -04:00
Nathan Hjelm
76320a8ba5 opal: rename opal_atomic_init to opal_atomic_lock_init
This function is used to initalize and opal atomic lock. The old name
was confusing.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2017-08-07 14:15:11 -06:00
Joshua Ladd
c27beea3a1 Merge pull request #3962 from karasevb/ucx_detect
configure: detect UCX support by default
2017-08-03 16:33:57 -04:00