1
1

29498 Коммитов

Автор SHA1 Сообщение Дата
Geoff Paulsen
811dfc63e0
Merge pull request #6550 from rhc54/cmr402/clnup
v4.0.x: Cleanup race condition in finalize that leads to incomplete vader cleanup
2019-04-09 10:13:15 -05:00
Geoff Paulsen
b3492090c2
Merge pull request #6561 from benmenadue/fix-scoll-fca
v4.0.x: scoll/fca: add missing argument to call to original broadcast
2019-04-08 14:05:47 -05:00
Geoff Paulsen
db1cb3002f
Merge pull request #6564 from benmenadue/v4.0.x-fix-shmem-context
v4.0.x: add missing #include to oshmem/shmem/c/shmem_context.c.
2019-04-08 14:00:04 -05:00
Mark Allen
36583df689 in-place conversion macro writes into INPUT argument
In fint_2_int.h there are some conversion macros for logicals. It has
one path for OMPI_SIZEOF_FORTRAN_LOGICAL != SIZEOF_INT where a new array
would be allocated and the conversions then might expand to
    c_array[i] = (array[i] == 0 ? 0 : 1)
and another path for OMPI_SIZEOF_FORTRAN_LOGICAL == SIZEOF_INT where it
does things "in place", so the same conversion there would just be
    array[i] = (array[i] == 0 ? 0 : 1)

The problem is some of the logical arrays being converted are INPUT
arguments. And it's possible for some compilers to even put the argument
in read-only memory so the above "in place" conversion SEGV's.  A
testcase I have used
    call MPI_CART_SUB(oldcomm, (/.true.,.false./), newcomm, ierr)
and gfortran put the second arg in read-only mem.

In cart_sub_f.c you can trace the ompi_fortran_logical_t *remain_dims arg.
remain_dims[] is for input only, but the file uses
    OMPI_LOGICAL_ARRAY_NAME_DECL(remain_dims);
    OMPI_ARRAY_LOGICAL_2_INT(remain_dims, ndims);
    PMPI_Cart_sub(..., OMPI_LOGICAL_ARRAY_NAME_CONVERT(remain_dims), ...);
    OMPI_ARRAY_INT_2_LOGICAL(remain_dims, ndims);
to convert it to c-ints make a C call then restore it to Fortran logicals
before returning.

It's not always wrong to convert purely in-place, eg cart_get_f.c has
a periods[] that's exclusively for OUTPUT and it would be fine with the
macros as they were. But I still say the macros are invalid because they
don't distinguish whether they're being used on INPUT or OUTPUT args and
thus they can't be used in a way that's legal for both cases.

It might be possible to fix the macros by adding more of them so that
cart_create_f.c and cart_get_f.c would use different macros that give
more context. But my fix here is just to turn off the first block and
make all paths run as if OMPI_SIZEOF_FORTRAN_LOGICAL != SIZEOF_INT.

The main macros that get enlarged by this change are
    define OMPI_ARRAY_LOGICAL_2_INT_ALLOC : mallocs now
    define OMPI_ARRAY_LOGICAL_2_INT : also mallocs now
But these are only used in 4 places, three of which are the purpose of
this checkin, to avoid the former in-place expansion of an INPUT arg:
    cart_create_f.c
    cart_map_f.c
    cart_sub_f.c
and one of which is an OUPUT arg that was fine and that gets
unnecessarily expanded into a separate array by this checkin.
    cart_get_f.c

So I think an unnecessary malloc in cart_get_f.c is the only downside
to this change, where the logicals array argument could have been used
and converted in place.

Signed-off-by: Mark Allen <markalle@us.ibm.com>

Update provided by Gilles Gouaillardet to keep the in-place option
if OMPI_FORTRAN_VALUE_TRUE == 1 where no conversion is needed.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
(cherry picked from commit 0a7f1e3cc58dabe536df00ae9c97f7e9d27103ad)
2019-04-05 13:34:09 -04:00
Howard Pritchard
702199f39e
Merge pull request #6545 from bertwesarg/v4.0.x-fix-cpp-condition
Fix use of bitwise operation in CPP condition (v4.0.x)
2019-04-05 07:58:09 -06:00
Ben Menadue
173192a6f4 Add missing #include to oshmem/shmem/c/shmem_context.c.
Signed-off-by: Ben Menadue <ben.menadue@nci.org.au>
(cherry picked from commit 063596b82837cf0b07926e2c696cd8f48a59143d)
2019-04-03 16:02:58 +11:00
Ben Menadue
001fa5b6ce Add missing nlong_type parameter to call to original broadcast in scoll/fca broadcast.
Signed-off-by: Ben Menadue <ben.menadue@nci.org.au>
2019-04-03 14:01:41 +11:00
Howard Pritchard
8261cdab06
Merge pull request #6554 from James-A-Clark/v4.0.x
Add a compilation flag that adds unwind info to all files that are present in the stack starting from MPI_Init (v4.0.x cherry pick)
2019-04-02 09:05:55 -06:00
Howard Pritchard
976cc1e07f
Merge pull request #6509 from janjust/oshmem-multiple-contexts-v4.0.x
v4.0.x: Oshmem multiple contexts
2019-04-01 13:15:38 -06:00
Howard Pritchard
b11cb23b71
Merge pull request #6519 from sam6258/int4_cswap_fix_v4.0.x
v4.0.x: shmem/fortran: Fix invalid datatype size in call to atomic cswap
2019-04-01 13:09:17 -06:00
James Clark
d8dc69feb5 Add a compilation flag that adds unwind info to all files that are present in the stack starting from MPI_Init.
This is so when a debugger attaches using MPIR, it can step out of this stack back into main.
This cannot be done with certain aggressive optimisations and missing debug information.

Signed-off-by: James Clark <james.clark@arm.com>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>

Co-authored-by: Jeff Squyres <jsquyres@cisco.com>

(cherry-picked from 20f5840)
2019-04-01 11:10:04 +01:00
Ralph Castain
2536b4f869 Remove stale ORTE code
Functionality moved to PMIx

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit cfdd08d309d9ebc48229f0ca68ceec64a7e6389f)
2019-03-31 11:26:18 -07:00
Ralph Castain
861016c3b2 Cleanup race condition in finalize
See https://github.com/open-mpi/ompi/issues/5798#issuecomment-426545893
for a lengthy explanation

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 57f6b94fa53166bc4d513be4507382e832a3a8c7)
2019-03-31 11:23:27 -07:00
Bert Wesarg
7f65e5b720 Fix use of bitwise operation in CPP condition
Signed-off-by: Bert Wesarg <bert.wesarg@tu-dresden.de>
(cherry picked from commit 18525ce39be78ea695ce51c64a6eb443a2dbd899)
2019-03-29 10:17:09 +01:00
Howard Pritchard
9a1b6cfc79
Merge pull request #6529 from hppritcha/topic/roll_to_v4.0.2a1
VERSION: roll to v4.0.2a1
2019-03-27 12:08:19 -06:00
Howard Pritchard
697437169a
Merge pull request #6528 from hppritcha/topic/minor_news_typo
NEWS: minor typo fix
2019-03-27 12:07:55 -06:00
Howard Pritchard
9e73e3e520 VERSION: roll to v4.0.2a1
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2019-03-27 11:20:05 -06:00
Howard Pritchard
812fd4aa2b NEWS: minor typo fix
[skip ci]

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2019-03-27 11:18:29 -06:00
Geoff Paulsen
b8a8ae9394
Merge pull request #6520 from gpaulsen/topic/v4.0.1/README_oops
Describing Issue 6114 with v4.0.0 in README.
2019-03-26 10:18:13 -05:00
Geoffrey Paulsen
176356249c README: Describes the now fixed Issue 6114
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
2019-03-26 10:12:15 -05:00
Scott Miller
5f4f5d45b3 shmem/fortran: Fix invalid datatype size in call to atomic cswap
Signed-off-by: Scott Miller <scott.miller1@ibm.com>
(cherry picked from commit 6b294e064150d26dfc68ec307cf4cd2e40891a1b)
2019-03-25 12:38:04 -04:00
Xin Zhao
69a80fce9f ompi/oshmem/spml/ucx: use lockfree array to optimize spml_ucx_progress/delete oshmem_barrier in shmem_ctx_destroy
ompi/oshmem/spml/ucx: optimize spml ucx progress

Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 9c3d00b144641d2929f830279dcc9d163c38e9e1)
2019-03-21 23:59:58 +02:00
Xin Zhao
580b584179 ompi/oshmem/spml/ucx:delete oob path of getting rkeys in spml ucx
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit e0414006b0c0a8e9918a4cf8ac4bb819b977ec91)
2019-03-21 23:59:46 +02:00
Xin Zhao
596997c194 ompi/oshmem/spml/ucx: defer clean up shmem_ctx to shmem_finalize
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit e1c1ab020227fc18d145379ab29ea86a3cdb66b1)
2019-03-21 23:58:23 +02:00
Geoff Paulsen
97aa434182
Merge pull request #6511 from gpaulsen/topic/v4.0.x/rc3
Update VERSION to v4.0.1rc3
2019-03-21 16:17:57 -05:00
Geoffrey Paulsen
8e04fb3633 Update VERSION to v4.0.1rc3
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
2019-03-21 16:38:51 -04:00
Xin Zhao
ce54b63b90 ompi/oshmem: add spml_context back to sshmem_type in memheap, to keep track of ucx_ctx_default's rkeys
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 48033ac1f43159c053241b65e74a39777e5e31e4)
2019-03-20 23:30:21 +02:00
Xin Zhao
06183a7bec ompi/oshmem/spml/ucx: let shmem_finalize to clean up any ctx left
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 9a060009622e9220d6332d9b63f6a1a7328418a0)
2019-03-20 23:30:09 +02:00
Xin Zhao
91793484ed OMPI/OSHMEM: bug-fix: store mkeys for each oshmem ctx.
Signed-off-by: Xin Zhao <xinz@mellanox.com>
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 289595e45dc3ebfe5ae1a9dc6f347b1b2d569c4a)
2019-03-20 23:29:53 +02:00
Xin Zhao
f666d75322 ompi/oshmem/spml/ucx: fix eps destroy in shmem_ctx_destroy().
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 79ba7526677bd1641239bb77559a2999c8cd3a4a)
2019-03-20 23:29:38 +02:00
Howard Pritchard
15cfba5347
Merge pull request #6503 from jjhursey/v4x-rm-hash-pmix3
Do not force 'hash' gds on direct modex
2019-03-19 17:58:26 -05:00
Geoff Paulsen
31ebbb2a8d
Merge pull request #6502 from nysal/v4.0.x_spinlock_fix
opal/atomics: Add acquire semantics back for spinlocks
2019-03-19 11:44:46 -05:00
Joshua Hursey
45526fadee Do not force 'hash' gds on direct modex
* Forcing the 'hash' gds component should not be necessary any more.

Port of PR #6498 (component names changed so a cherry-pick would not work)

Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
2019-03-19 10:52:17 -05:00
Nysal Jan K.A
1329cef213 opal/atomics: Add acquire semantics back for spinlocks
This was introduced in commit 9d0b3fe9

Signed-off-by: Nysal Jan K.A <jnysal@in.ibm.com>
(cherry picked from commit 00f27a80fc63053db1aeb42140148d7a3d1379b3)
2019-03-19 19:45:20 +05:30
Geoff Paulsen
6cb00aa333
Merge pull request #6499 from hppritcha/topic/news_updates_for_4.0.1rc2
NEWS: add a few news items for 4.0.1rc2
2019-03-19 05:22:40 -05:00
Howard Pritchard
ce013130cb NEWS: add a few news items for 4.0.1rc2
a little late, but a couple of bullets for the
4.0.1rc2 NEWS.

[skip ci]

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2019-03-19 04:13:32 -06:00
Gilles Gouaillardet
8da4605589 btl/openib: immediately release the device when no port is allowed
Many thanks to Sergey Oblomov for reporting this issue
and the countless traces provided when troubleshooting it.

This is a one-off commit for the v4.0.x branch since btl/openib has been removed
 from master.

Refs. open-mpi/ompi#6137

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2019-03-19 09:26:11 +09:00
Gilles Gouaillardet
c58c774981 btl/openib: have add_proc() return immediately when the port is disabled.
Fixes an issue introduced in open-mpi/ompi@0a2ce58040

This is a one-off commit for the v4.0.x branch since btl/openib has been removed from master.

Refs. open-mpi/ompi#6137

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2019-03-19 09:24:25 +09:00
Gilles Gouaillardet
d7053a306a btl/openib: delay UCX warning to add_procs()
If UCX is available, then pml/ucx will be used instead of
pml/ob1 + btl/openib, so there is no need to warn about
btl/openib not supporting Infiniband.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>

(cherry picked from commit open-mpi/ompi@0a2ce58040)
2019-03-19 09:24:00 +09:00
Geoff Paulsen
efcbc13d2f
Merge pull request #6496 from gpaulsen/v4.0.x
Reving to v4.0.1rc2
2019-03-18 16:34:16 -05:00
Geoffrey Paulsen
2ae9a8a3d6 Reving to v4.0.1rc2
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
2019-03-18 16:33:26 -05:00
Howard Pritchard
ceb93d7c03
Merge pull request #6491 from bosilca/v4.0.x
v4.0.x: Cherry-pick fixes for issue #6258 from master (vader fixes)
2019-03-15 17:08:52 -06:00
Howard Pritchard
27899b0e8f
Merge pull request #6486 from hoopoepg/topic/check-ucx-params-v4.0
PML/SPML/UCX: added evaluation of mmap events - v4.0
2019-03-14 17:02:46 -06:00
Howard Pritchard
27c0e95b01
Merge pull request #6489 from markalle/v4.0.x
v4.0.x: opal_hwloc_base_cset2str() off-by-1 in its strncat()
2019-03-14 17:00:42 -06:00
Nathan Hjelm
3df8ed9cc0
btl/vader: fix fragment sizes used by free lists
This commit fixes a bug introduced in
f62d26ddbc8cda4d985cceee531a2ec32406d1f6. That commit changed how
vader allocates fragment memory from the shared memory
segment. Unfortunately, the values used for the fragment sizes did not
include space for the fragment header. This can cause an overrun of
data from one fragment to the header of the next fragment.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2019-03-14 17:25:31 -04:00
Nathan Hjelm
20017d345e
btl/vader: use basic mpool type to handle frag/fbox allocation
This commit updates btl/vader to use an mpool for handling all shared
memory allocations (frags, fboxes).

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2019-03-14 17:21:12 -04:00
Nathan Hjelm
bac6024b5a
mpool: add new base module type "basic"
This commit adds a new mpool base module type: basic. This module can
be used with an opal_free_list_t to allocate space from a
pre-allocated block (such as a shared memory region). The new module
only supports allocation and is not meant for more dynamic use cases
at this time.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2019-03-14 17:20:30 -04:00
Mark Allen
fcf53becc3 opal_hwloc_base_cset2str() off-by-1 in its strncat()
I think the strncat() calls here need to be of the form
    strncat(str, new_str_to_add, len - strlen(new_str_to_addstr) - 1);
since in the OMPI calls len is being used as total number of bytes
in str.

strncat(dest,src,n) on the other hand is documented as writing up to
n chars from the incoming string plus 1 for the null, for n+1 total
bytes it can write.

Signed-off-by: Mark Allen <markalle@us.ibm.com>
(cherry picked from commit 30d60994d258f5f0b7c432efd284d1b6b8333faf)

Conflicts:
	opal/mca/hwloc/base/hwloc_base_util.c
2019-03-14 13:08:25 -04:00
Sergey Oblomov
bed8141088 COMMON/UCX: rewording of hooks suggestion
- also updated output macro

Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
(cherry picked from commit c319cf9adefb69c78a73eb4a83a40dee5b697a53)
2019-03-14 16:48:36 +02:00
Sergey Oblomov
14c271f993 PML/SPML/UCX: added evaluation of mmap events
- there was a set of UCX related issues reported which caused
  by mmap API hooks conflicts. We added diagnostic of such
  problems to simplify bug-resolving pipeline

Signed-off-by: Sergey Oblomov <sergeyo@mellanox.com>
(cherry picked from commit d8e3562bae700d84873c1d5ca9c45c846d7387ed)
2019-03-14 16:48:25 +02:00