1
1

29524 Коммитов

Автор SHA1 Сообщение Дата
Geoff Paulsen
5d4c9b444a
Merge pull request #6636 from mwheinz/REF6638-v4.0.x
v4.0.x buildrpm.sh no longer respects the value of rpmtopdir
2019-05-07 09:41:14 -05:00
Geoff Paulsen
73f9bcc374
Merge pull request #6632 from brminich/topic/shmem_all2all_put_4.0.x
SPML/UCX: Add shmemx_alltoall_global_nb routine to shmemx.h 4.0.x
2019-05-07 08:05:01 -05:00
Howard Pritchard
8e968f16a6
Merge pull request #6626 from ggouaillardet/topic/v4.0.x/mpi_combiner_xyz_integer
v4.0.x: mpi: mark MPI_COMBINER_{HVECTOR,HINDEXED,STRUCT}_INTEGER removed
2019-05-04 07:25:40 -06:00
Michael Heinz
77caeb9bfa Corrects some whitespace issues with buildrpm.sh
Signed-off-by: Michael Heinz <michael.william.heinz@intel.com>
(cherry picked from commit 8562211623cd7501c13b1d3976f7fe0a3d27c91d)
2019-05-03 09:54:41 -04:00
Michael Heinz
94e842bb34 buildrpm.sh no longer respects the value of rpmtopdir
In OMPI 2.1.2, buildrpm.sh could work with a value of rpmtopdir that was
set in the environment. In newer versions this is no longer true,
causing such values to be ignored. This patch adds a new argument to
buildrpm.sh, -R, which allows the user to specify where to build the
RPMs.

Signed-off-by: Michael Heinz <michael.william.heinz@intel.com>
(cherry picked from commit 687a5603a122af332bb939bba13aec78beb206cd)
2019-05-03 09:54:33 -04:00
George Bosilca
48f824327c Fix the leak of fragments for persistent sends.
The rdma_frag attached to the send request was not correctly released
upon request completion, leaking until MPI_Finalize. A quick solution
would have been to add RDMA_FRAG_RETURN at different locations on the
send request completion, but it would have unnecessarily made the
sendreq completion path more complex. Instead, I added the length to
the RDMA fragment so that it can be completed during the remote ack.

Be more explicit on the comment.

The rdma_frag can only be freed once when the peer forced a protocol
change (from RDMA GET to send/recv). Otherwise the fragment will be
returned once all data pertaining to it has been trasnferred.

NOTE: Had to add a typedef for "opal_atomic_size_t" from master into
opal/threads/thread_usage.h into this cherry pick (it is in
opal/include/opal_stdatomic.h on master, but that file does not exist
here on the v4.0.x branch).

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
(cherry picked from commit a16cf0e4dd6df4dea820fecedd5920df632935b8)
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2019-05-03 06:20:02 -07:00
Brelle Emmanuel
c44821aef5 pml/ob1: fixed local handle sent during PUT control message
In case of using a btl_put in ob1, the handle of the locally registered
memory is sent with a PUT control message. In the current master code
the sent handle is necessary the handle in the frag but if the handle
has been successfully registered in the request, the frag structure does
not have any valid handle and all fragments use the request one.

I suggest to check if the handle in the fragment is valid and if not to
send the handle from the request.

Signed-off-by: Brelle Emmanuel <emmanuel.brelle@atos.net>
(cherry picked from commit e630046a4b82bc01379fb055af4c0e414c2a8e8f)
2019-05-03 05:53:35 -07:00
Mikhail Brinskii
6861a68de6 SPML/UCS: CR comments p2
Signed-off-by: Mikhail Brinskii <mikhailb@mellanox.com>
(cherry picked from commit d4843b165172a9cffbdc967ec92ec1d60d053903)
2019-05-02 21:27:15 +03:00
Mikhail Brinskii
1c56f49a44 SPML/UCX: CR comments p1
Signed-off-by: Mikhail Brinskii <mikhailb@mellanox.com>
(cherry picked from commit c4c99457db6577f85b5b1c4c6849b5a04c226a84)
2019-05-02 21:26:55 +03:00
Mikhail Brinskii
e4ee56d1f3 SPML/UCX: Add shmemx_alltoall_global_nb routine to shmemx.h
The new routine transfers the data asynchronously from the source PE to all
PEs in the OpenSHMEM job. The routine returns immediately. The source and
target buffers are reusable only after the completion of the routine.
After the data is transferred to the target buffers, the counter object
is updated atomically. The counter object can be read either using atomic
operations such as shmem_atomic_fetch or can use point-to-point synchronization
routines such as shmem_wait_until and shmem_test.

Signed-off-by: Mikhail Brinskii <mikhailb@mellanox.com>
(cherry picked from commit 2ef5bd8b3671f1e10caf00d06d66d120eac9c5be)
2019-05-02 21:25:59 +03:00
Howard Pritchard
3cafd02c7f
Merge pull request #6572 from markalle/v40x_fortran_macro
in-place conversion macro writes into INPUT argument
2019-05-01 11:54:12 -06:00
Howard Pritchard
41ef5c7a10
Merge pull request #6594 from vspetrov/osc_ucx_rget_rkey_fix
OSC/UCX: use correct rkey for atomic_fadd in rget/rput
2019-05-01 11:53:17 -06:00
Howard Pritchard
ca5d58f955
Merge pull request #6615 from markalle/merge_v40x_romio
fixing an unsafe usage of integer disps[] (romio321 gpfs)
2019-05-01 11:51:53 -06:00
Gilles Gouaillardet
e2638dbbf2 mpi: mark MPI_COMBINER_{HVECTOR,HINDEXED,STRUCT}_INTEGER removed
unless configure'd with --enable-mpi1-compatibility

This is a one-off commit for the v4.0.x branch since these symbols were
simply removed from master.

Thanks Lisandro Dalcin for reporting this.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2019-05-01 10:50:57 +09:00
Howard Pritchard
888d014590
Merge pull request #6624 from mwheinz/v4.0.x
v4.0.x: make-authors.pl script not compatible with being a submodule.
2019-04-30 07:55:24 -06:00
Michael Heinz
191c7f01a2 make-authors.pl script not compatible with being a submodule.
make-authors.pl checks that .git exists and is a directory before
getting the git log - but when a repo is checked out as a submodule of a
larger repository, .git is not a directory, it's just a text file.  This
can cause make-authors.pl to terminate inappropriately.

Author: Michael Heinz <michael.william.heinz@intel.com>
Signed-off-by: Michael Heinz <michael.william.heinz@intel.com>
(cherry picked from commit 0a8fa5439c626c01a68fd9cebda8f00597500f51)
2019-04-29 10:14:53 -04:00
Mark Allen
c081757462 fixing an unsafe usage of integer disps[] (romio321 gpfs)
There are a couple MPI_Alltoallv calls in ad_gpfs_aggrs.c where the
send/recv data comes from places like req[r].lens, and the send
buffer and send displacements for example were being calculated as
    sbuf = pick one of the reqs: req[bottom].lens
    sdisps[r] = req[r].lens - req[bottom].lens
which might be okay if the .lens was data inside of req[] so they'd
all be close to each other. But each .lens field is just a pointer
that's malloced, so those addresses can be all over the place, so the
integer-sized sdisps[] isn't safe.

I changed it to have a new extra array sbuf and rbuf for those two
Alltoallv calls, and copied the data into the sbuf from the same
locations it used to be setting up the sdisps[] at, and after the
Alltoallv I copy the data out of the new rbuf into the same
locations it used to be setting up the rdisps[] at.

For what it's worth I was able to get this to fail -np 2 on a GPFS
filesystem with hints romio_cb_write enable. I didn't whittle the
test down to something small, but it was failing in an
MPI_File_write_all call.

Signed-off-by: Mark Allen <markalle@us.ibm.com>
(cherry picked from commit d85cac8f1a11495415b67ecab69d2ae1cd19d155)
2019-04-25 14:22:19 -04:00
Howard Pritchard
f3edfaa2ac
Merge pull request #6610 from jsquyres/pr/ob1-get-frag-fail-fix
v4.0.x: ob1 get_frag fail fix
2019-04-23 09:27:08 -06:00
Brelle Emmanuel
2a4bc0cb58 pml/ob1: fixed exit from get_frag_fail when falling back on btl_put
In the case the btl_get fails Ob1 tries to fallback on btl_put first but
the return code was ignored. So the code fell back on both btl_put and
btl_send.

Signed-off-by: Brelle Emmanuel <emmanuel.brelle@atos.net>
(cherry picked from commit 9c689f2225d29aa152627f39bab841afead254af)
2019-04-22 14:25:34 -07:00
Howard Pritchard
9b73e8a7c0
Merge pull request #6600 from vspetrov/v4.0.x_osc_ucx_no_op_null_addr_handling
OSC/UCX: correctly handle NULL origin addr and MPI_NO_OP
2019-04-18 15:36:13 -06:00
Valentin Petrov
2947ab2dbc OSC/UCX: correctly handle NULL origin addr and MPI_NO_OP
Signed-off-by: Valentin Petrov <valentinp@mellanox.com>
2019-04-17 10:35:34 +03:00
Howard Pritchard
9f7b41f588
Merge pull request #6585 from vspetrov/v4.0.x
V4.0.x Fixes the O(N^2) loop in the mca_scoll_mpi_comm_query
2019-04-16 09:10:55 -06:00
Valentin Petrov
68c88e86f2 OSC/UCX: use correct rkey for atomic_fadd in rget/rput
Signed-off-by: Valentin Petrov <valentinp@mellanox.com>
2019-04-16 15:24:57 +03:00
Valentin Petrov
281f78c6e4 Fixes the O(N^2) loop in the mca_scoll_mpi_comm_query
The new proc group is created from the "world_group" based on the
      ranks mapping which can be directly taken from proc_name->vpid.

Signed-off-by: Valentin Petrov <valentinp@mellanox.com>
2019-04-15 08:43:09 +03:00
Howard Pritchard
5d6657ea40
Merge pull request #6582 from jsquyres/pr/v4.0.x/fix-overtake
v4.0.x: pml/ob1: fix deadlock with communicator flag ALLOW_OVERTAKE.
2019-04-10 10:46:09 -06:00
Thananon Patinyasakdikul
5999fdad5a pml/ob1: fix deadlock with communicator flag ALLOW_OVERTAKE.
We missed an assert to check if ALLOW_OVERTAKE is set or not before
validating the sequence number and this will cause deadlock.

Signed-off-by: Thananon Patinyasakdikul <tpatinya@utk.edu>
(cherry picked from commit 0263456cf4e99efc67d38acd100cf948e0399d63)
2019-04-09 11:24:24 -07:00
Geoff Paulsen
811dfc63e0
Merge pull request #6550 from rhc54/cmr402/clnup
v4.0.x: Cleanup race condition in finalize that leads to incomplete vader cleanup
2019-04-09 10:13:15 -05:00
Geoff Paulsen
b3492090c2
Merge pull request #6561 from benmenadue/fix-scoll-fca
v4.0.x: scoll/fca: add missing argument to call to original broadcast
2019-04-08 14:05:47 -05:00
Geoff Paulsen
db1cb3002f
Merge pull request #6564 from benmenadue/v4.0.x-fix-shmem-context
v4.0.x: add missing #include to oshmem/shmem/c/shmem_context.c.
2019-04-08 14:00:04 -05:00
Mark Allen
36583df689 in-place conversion macro writes into INPUT argument
In fint_2_int.h there are some conversion macros for logicals. It has
one path for OMPI_SIZEOF_FORTRAN_LOGICAL != SIZEOF_INT where a new array
would be allocated and the conversions then might expand to
    c_array[i] = (array[i] == 0 ? 0 : 1)
and another path for OMPI_SIZEOF_FORTRAN_LOGICAL == SIZEOF_INT where it
does things "in place", so the same conversion there would just be
    array[i] = (array[i] == 0 ? 0 : 1)

The problem is some of the logical arrays being converted are INPUT
arguments. And it's possible for some compilers to even put the argument
in read-only memory so the above "in place" conversion SEGV's.  A
testcase I have used
    call MPI_CART_SUB(oldcomm, (/.true.,.false./), newcomm, ierr)
and gfortran put the second arg in read-only mem.

In cart_sub_f.c you can trace the ompi_fortran_logical_t *remain_dims arg.
remain_dims[] is for input only, but the file uses
    OMPI_LOGICAL_ARRAY_NAME_DECL(remain_dims);
    OMPI_ARRAY_LOGICAL_2_INT(remain_dims, ndims);
    PMPI_Cart_sub(..., OMPI_LOGICAL_ARRAY_NAME_CONVERT(remain_dims), ...);
    OMPI_ARRAY_INT_2_LOGICAL(remain_dims, ndims);
to convert it to c-ints make a C call then restore it to Fortran logicals
before returning.

It's not always wrong to convert purely in-place, eg cart_get_f.c has
a periods[] that's exclusively for OUTPUT and it would be fine with the
macros as they were. But I still say the macros are invalid because they
don't distinguish whether they're being used on INPUT or OUTPUT args and
thus they can't be used in a way that's legal for both cases.

It might be possible to fix the macros by adding more of them so that
cart_create_f.c and cart_get_f.c would use different macros that give
more context. But my fix here is just to turn off the first block and
make all paths run as if OMPI_SIZEOF_FORTRAN_LOGICAL != SIZEOF_INT.

The main macros that get enlarged by this change are
    define OMPI_ARRAY_LOGICAL_2_INT_ALLOC : mallocs now
    define OMPI_ARRAY_LOGICAL_2_INT : also mallocs now
But these are only used in 4 places, three of which are the purpose of
this checkin, to avoid the former in-place expansion of an INPUT arg:
    cart_create_f.c
    cart_map_f.c
    cart_sub_f.c
and one of which is an OUPUT arg that was fine and that gets
unnecessarily expanded into a separate array by this checkin.
    cart_get_f.c

So I think an unnecessary malloc in cart_get_f.c is the only downside
to this change, where the logicals array argument could have been used
and converted in place.

Signed-off-by: Mark Allen <markalle@us.ibm.com>

Update provided by Gilles Gouaillardet to keep the in-place option
if OMPI_FORTRAN_VALUE_TRUE == 1 where no conversion is needed.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
(cherry picked from commit 0a7f1e3cc58dabe536df00ae9c97f7e9d27103ad)
2019-04-05 13:34:09 -04:00
Howard Pritchard
702199f39e
Merge pull request #6545 from bertwesarg/v4.0.x-fix-cpp-condition
Fix use of bitwise operation in CPP condition (v4.0.x)
2019-04-05 07:58:09 -06:00
Ben Menadue
173192a6f4 Add missing #include to oshmem/shmem/c/shmem_context.c.
Signed-off-by: Ben Menadue <ben.menadue@nci.org.au>
(cherry picked from commit 063596b82837cf0b07926e2c696cd8f48a59143d)
2019-04-03 16:02:58 +11:00
Ben Menadue
001fa5b6ce Add missing nlong_type parameter to call to original broadcast in scoll/fca broadcast.
Signed-off-by: Ben Menadue <ben.menadue@nci.org.au>
2019-04-03 14:01:41 +11:00
Howard Pritchard
8261cdab06
Merge pull request #6554 from James-A-Clark/v4.0.x
Add a compilation flag that adds unwind info to all files that are present in the stack starting from MPI_Init (v4.0.x cherry pick)
2019-04-02 09:05:55 -06:00
Howard Pritchard
976cc1e07f
Merge pull request #6509 from janjust/oshmem-multiple-contexts-v4.0.x
v4.0.x: Oshmem multiple contexts
2019-04-01 13:15:38 -06:00
Howard Pritchard
b11cb23b71
Merge pull request #6519 from sam6258/int4_cswap_fix_v4.0.x
v4.0.x: shmem/fortran: Fix invalid datatype size in call to atomic cswap
2019-04-01 13:09:17 -06:00
James Clark
d8dc69feb5 Add a compilation flag that adds unwind info to all files that are present in the stack starting from MPI_Init.
This is so when a debugger attaches using MPIR, it can step out of this stack back into main.
This cannot be done with certain aggressive optimisations and missing debug information.

Signed-off-by: James Clark <james.clark@arm.com>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>

Co-authored-by: Jeff Squyres <jsquyres@cisco.com>

(cherry-picked from 20f5840)
2019-04-01 11:10:04 +01:00
Ralph Castain
2536b4f869 Remove stale ORTE code
Functionality moved to PMIx

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit cfdd08d309d9ebc48229f0ca68ceec64a7e6389f)
2019-03-31 11:26:18 -07:00
Ralph Castain
861016c3b2 Cleanup race condition in finalize
See https://github.com/open-mpi/ompi/issues/5798#issuecomment-426545893
for a lengthy explanation

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 57f6b94fa53166bc4d513be4507382e832a3a8c7)
2019-03-31 11:23:27 -07:00
Bert Wesarg
7f65e5b720 Fix use of bitwise operation in CPP condition
Signed-off-by: Bert Wesarg <bert.wesarg@tu-dresden.de>
(cherry picked from commit 18525ce39be78ea695ce51c64a6eb443a2dbd899)
2019-03-29 10:17:09 +01:00
Howard Pritchard
9a1b6cfc79
Merge pull request #6529 from hppritcha/topic/roll_to_v4.0.2a1
VERSION: roll to v4.0.2a1
2019-03-27 12:08:19 -06:00
Howard Pritchard
697437169a
Merge pull request #6528 from hppritcha/topic/minor_news_typo
NEWS: minor typo fix
2019-03-27 12:07:55 -06:00
Howard Pritchard
9e73e3e520 VERSION: roll to v4.0.2a1
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2019-03-27 11:20:05 -06:00
Howard Pritchard
812fd4aa2b NEWS: minor typo fix
[skip ci]

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2019-03-27 11:18:29 -06:00
Geoff Paulsen
b8a8ae9394
Merge pull request #6520 from gpaulsen/topic/v4.0.1/README_oops
Describing Issue 6114 with v4.0.0 in README.
2019-03-26 10:18:13 -05:00
Geoffrey Paulsen
176356249c README: Describes the now fixed Issue 6114
Signed-off-by: Geoffrey Paulsen <gpaulsen@us.ibm.com>
2019-03-26 10:12:15 -05:00
Scott Miller
5f4f5d45b3 shmem/fortran: Fix invalid datatype size in call to atomic cswap
Signed-off-by: Scott Miller <scott.miller1@ibm.com>
(cherry picked from commit 6b294e064150d26dfc68ec307cf4cd2e40891a1b)
2019-03-25 12:38:04 -04:00
Xin Zhao
69a80fce9f ompi/oshmem/spml/ucx: use lockfree array to optimize spml_ucx_progress/delete oshmem_barrier in shmem_ctx_destroy
ompi/oshmem/spml/ucx: optimize spml ucx progress

Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit 9c3d00b144641d2929f830279dcc9d163c38e9e1)
2019-03-21 23:59:58 +02:00
Xin Zhao
580b584179 ompi/oshmem/spml/ucx:delete oob path of getting rkeys in spml ucx
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit e0414006b0c0a8e9918a4cf8ac4bb819b977ec91)
2019-03-21 23:59:46 +02:00
Xin Zhao
596997c194 ompi/oshmem/spml/ucx: defer clean up shmem_ctx to shmem_finalize
Signed-off-by: Tomislav Janjusic <tomislavj@mellanox.com>
(cherry picked from commit e1c1ab020227fc18d145379ab29ea86a3cdb66b1)
2019-03-21 23:58:23 +02:00