Fine tuning of flux component
Fix a few minor issues with the initial cut:
* Job id could be obtained from the PMI kvsname like SLURM,
but simpler to getenv (FLUX_JOB_ID)
* Flux pmi-1 doesn't define PMI_BOOL, PMI_TRUE, PMI_FALSE
* Flux pmi-1 maps the deprecated PMI_Get_kvs_domain_id() to
PMI_KVS_Get_my_name() internally, so just call that instead.
* Drop residual slurm references.
Add wrappers for PMI functions so that if HAVE_FLUX_PMI_LIBRARY
is not defined, the component can dlopen libpmi.so at location
specified by the FLUX_PMI_LIBRARY_PATH env variable, which adds
flexibility. If HAVE_FLUX_PMI_LIBRARY is defined, link with
libpmi.so at build time in the usual way.
Update configury for flux component
Update m4 so the configure options work as follows:
--with-flux-pmi
Build Flux PMI support (default: yes)
--with-flux-pmi-library
Link Flux PMI support with PMI library at build
time. Otherwise the library is opened at runtime at
location specified by FLUX_PMI_LIBRARY_PATH environment
variable. Use this option to enable Flux support when
building statically or without dlopen support (default: no)
If the latter option is provided, the library/header is located at
build time using the pkg-config module 'flux-pmi'. Otherwise there
is no library/header dependency.
Handle the case where ompi is configured with --disable-dlopen
or --enable-statkc. In those cases, don't build the component
unless --with-flux-pmi-library is provided.
It is fatal if the user explicitly requests --with-flux-pmi but
it cannot be built (e.g. due to --disable-dlopen).
Add a schizo/flux component
Update schizo/flux component
Eliminate slurm-specific usage cases.
Since the module is only loaded if FLUX_JOB_ID is set, there are
only two cases to handle:
1) App was launched indirectly through mpirun. This is not yet
supported with Flux, but hook remains in case this mode is supported
in the future.
2) App was launched directly by Flux, with Flux providing
CPU binding, if any.
Fix up white space in pmix/flux component
Drop non-blocking fence from pmix:flux component
The flux PMI-1 library is not thread safe, therefore
register a regular blocking fence callback instead of the
thread-shifting fencenb().
pmix/flux component avoids extra PMI_KVS_Gets
Keys stored into the base cache under the wildcard
rank are not intended to be part of the global key namespace.
These keys therefore should not trigger a PMI_KVS_Get() if they
are not found in the cache.
Minor pmix/flux component cleanup
pmix/flux: drop code for fetching unused pmix_id
pmix/flux: err_exit must return error
Problem: in flux_init(), although 'ret' (variable holding
err_exit return code) is initialized to OPAL_ERROR, the
variable is reused as a temporary result code, so if there are
some successes followed by a failure that doesn't set 'ret',
flux_init() could return success with PMI not initialized.
Ensure that a "goto err_exit" returns OPAL_ERROR if 'ret'
is not set to some other error code.
pmix/flux: don't mix OPAL_ and PMI_ return codes
Problem: flux_init() can return both PMI_ and OPAL_ return
codes. Although OPAL_SUCCESS and PMI_SUCCESS are both defined
as 0, other codes are not compatible.
Ensure that flux_init() consistently uses 'rc' for PMI_
return codes and 'ret' for OPAL_ return codes.
pmix/flux: factor out repeated code for cache put
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
Newer x86 processors have a core invariant tsc. On these systems it is
safe to use the rtdtsc instruction as a monotonic timer. This commit
adds a new function to the opal timer code to check if the timer
backend is monotonic. On x86 it checks the appropriate bit and on
other architectures it parrots back the OPAL_TIMER_MONOTONIC value.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
* When using `MPI_Put` with `MPI_Win_lock_all` a hang is possible since
the `put` is waiting on `eager_send_active` to become `true` but
that variable might not be reset in the case of `MPI_Win_lock_all`
depending on other incoming events (e.g., `post` or ACKs of lock
requests.
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
* When using `MPI_Lock`/`MPI_Unlock` with `MPI_Get` and non-contiguous
datatypes is is possible that the unlock finishes too early before
the data is actually present in the recv buffer.
* We need to wait for the irecv to complete before unlocking the target.
This commit waits for the outgoing fragment counts to become equal
before unlocking.
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
* If the user uses PSCW synchronization after a Fence then the previous
epoch is not reset which can cause the PSCW to transfer data before
it is ready leading to wrong answers.
* This commit resets the `eager_send_active` in the start call.
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
Update ORTE support for dynamic PMIx operations e.g., PMIx_Spawn
Update to track master
Ensure that --disable-pmix-dstore actually disables the dstore. Sync to a few debugger updates
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
By default use mmap() to allocate memory for the symmetric heap.
It is safer and more portable choice than sysv and verbs.
Signed-off-by: Alex Mikheev <alexm@mellanox.com>
This commit fixes errors in the lb and extent of darray datatypes. For
these datatypes the lb should be the start offset of the rank's data
in the array and the extent should be the size of the entire
datatype. In master the lb was always 0 and the extent was always to
small. This commit updates the call to opal_datatype_resize to set the
correct lb and fixes the extent calculation.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
According to the MPI-3.1 p.52 and p.53 (cited below), a request
created by `MPI_*_INIT` but not yet started by `MPI_START` or
`MPI_STARTALL` is inactive therefore `MPI_WAIT` or its friends
must return immediately if such a request is passed.
The current implementation hangs in `MPI_WAIT` and its friends
in such case because a persistent request is initialized as
`req_complete = REQUEST_PENDING`. This commit fixes the
initialization.
Also, this commit fixes internal requests used in `MPI_PROBE`
and `MPI_IPROBE` which was marked wrongly as persistent.
MPI-3.1 p.52:
We shall use the following terminology: A null handle is a handle
with value MPI_REQUEST_NULL. A persistent request and the handle
to it are inactive if the request is not associated with any ongoing
communication (see Section 3.9). A handle is active if it is neither
null nor inactive. An empty status is a status which is set to return
tag = MPI_ANY_TAG, source = MPI_ANY_SOURCE, error = MPI_SUCCESS, and
is also internally configured so that calls to MPI_GET_COUNT,
MPI_GET_ELEMENTS, and MPI_GET_ELEMENTS_X return count = 0 and
MPI_TEST_CANCELLED returns false. We set a status variable to empty
when the value returned by it is not significant. Status is set in
this way so as to prevent errors due to accesses of stale information.
MPI-3.1 p.53:
One is allowed to call MPI_WAIT with a null or inactive request
argument. In this case the operation returns immediately with empty
status.
Signed-off-by: KAWASHIMA Takahiro <t-kawashima@jp.fujitsu.com>
instead of compilation date __DATE__, use a MPI_Get_library_version() like string
Thanks Alastair McKinstry for the report
Fixesopen-mpi/ompi#2518
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
