send reply before posting the receive request again to limit the recursion size to
number of receive requests.
send can call opal_progress which calls this function again. If recv req is started
stack size will be proportional to number of job ranks.
We recognize that this means other users of OPAL will need to "wrap" the opal_process_name_t if they desire to abstract it in some fashion. This is regrettable, and we are looking at possible alternatives that might mitigate that requirement. Meantime, however, we have to put the needs of the OMPI community first, and are taking this step to restore hetero and SPARC support.
Force completion of all puts before deregestering memheap/bss memory
Fixes a possible race condition where put request completion callback
is called when request context is already cleared.
Change-Id: I7ed887ec0b03a66ce5d3076a7edcf64061f57370
Do not allow combination of transports that is not compliant with
shmem spec. Especially do not allow mix of hw and software atomic
ops
Issue: 4721
Change-Id: Ide382f7510495df3d385f2a5ae5f9def6ef5332c
Use experimental verbs to allocate memory at fixed base
virtual address.
verbs will disqualify itself if shared_mr is disabled
or not supported and it is impossible to allocate memory
starting at fixed base virtual address.
verbs contig pages allocator did not guarantee fixed va, now it does.
(cherry picked from commit fd77ebd452)
Apply Jeff`s comments
Update with Jeff commits
(cherry picked from commit open-mpi/ompi-release@4dc487fc3d)
It looks like using MAP_PRIVATE instead of MAP_SHARED greatly
speeds up infiniband memory registration.
Change-Id: Id7089f58458ef8fff4034a2c4707d31f7e8b6694
Use experimental verbs to allocate memory at fixed base
virtual address.
verbs will disqualify itself if shared_mr is disabled
or not supported and it is impossible to allocate memory
starting at fixed base virtual address.
verbs contig pages allocator did not guarantee fixed va, now it does.
Added use_hp flag in sshmem/sysv variable to control huge page usage;
Added shared_mr sshmem/verbs;
Both paraemetes are set in auto.
Fix help messages
fixed by Igor, reviewed by @miked-mellanox and @alex-mikheev
Lock server pe computation was incorrect in cases when:
lock virtual address is signed long. In this case negative pe
value was returned.
In case when lock has different virtual adresses on different pes.
It can happen when memheap or static segment have different base
addresses. Use offset instead of absolute virtual address to
compute server pe
Fixed by @alex-mikheev, reviewed by @miked-mellanox
WHAT: Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL
All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies. This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP. Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose. UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs. A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.
This commit was SVN r32317.
OpenSHMEMspec 1.1 introduces a set of environment variables that allows users to configure the Open-SHMEM implementation, and receive information about the implementation.
- Add SMA_SYMMETRIC_SIZE - number of bytes to allocate for symmetric heap
- SHMEM_SYMMETRIC_HEAP_SIZE (Mellanox extension) is used by a user to provide a size of symmetric area. This change sets this env variable in case a user does not set this variable
directly.
fixed by Igor, reviewed by Miked
cmr=v1.8.2:reviwer=ompi-rm1.8
This commit was SVN r32257.
mca_btl_base_segment_t and replace them with des_local and des_remote
This change also updates the BTL version to 3.0.0. This commit does
not represent the final version of BTL 3.0.0. More changes are coming.
In making this change I updated all of the BTLs as well as BTL user's
to use the new structure members. Please evaluate your component to
ensure the changes are correct.
RFC text:
This is the first of several BTL interface changes I am proposing for
the 1.9/2.0 release series.
What: Change naming of btl descriptor members. I propose we change
des_src and des_dst (and their associated counts) to be des_local and
des_remote. For receive callbacks the des_local member will be used to
communicate the segment information to the callback. The proposed change
will include updating all of the doxygen in btl.h as well as updating
all BTLs and BTL users to use the new naming scheme.
Why: My btl usage makes use of both put and get operations on the same
descriptor. With the current naming scheme I need to ensure that there
is consistency beteen the segments described in des_src and des_dst
depending on whether a put or get operation is executed. Additionally,
the current naming prevents BTLs that do not require prepare/RMA matched
operations (do not set MCA_BTL_FLAGS_RDMA_MATCHED) from executing
multiple simultaneous put AND get operations. At the moment the
descriptor can only be used with one or the other. The naming change
makes it easier for BTL users to setup/modify descriptors for RMA
operations as the local segment and remote segment are always in the
same member field. The only issue I forsee with this change is that it
will require a little more work to move BTL fixes to the 1.8 release
series.
This commit was SVN r32196.
