1. orte_enable_recovery - default recovery policy, can be overridden on a per-job basis
2. orte_max_local_restarts - default max number of local restarts, can be overridden
3. orte_max_global_restarts - default max number of relocates, can be overridden
Implement the restart_proc API for the ODLS framework, reorganize the default fns a little to avoid copying code.
This commit was SVN r23057.
Note that this isn't a problem since MPI_Abort and orte_abort are only called under controlled circumstances - i.e., we are doing an orderly abort and not segfaulting. If we can't get the message out for some reason, then too bad - we'll still see an abnormal process termination and act accordingly.
This commit was SVN r23045.
1. file activity - can monitor file size, access and modification times. If these fail to change over a specified number of sampling iterations (rate is an mca param), then the errmgr is notified.
2. memory usage - checks amount of memory used by a process. Limit and sampling rate can be set.
This support must be enabled by configuring --enable-sensors.
ompi_info and orte-info have been updated to include the new framework.
Also includes some initial steps toward restoring the recovery capability. Most notably, the ODLS API has been extended to include a "restart_proc" entry for restarting a local process, and organizes the various ERRMGR framework globals into a single struct as we do in the other ORTE frameworks. Fix an oversight in the ERRMGR framework where a pointer array was constructed, but not initialized.
Implementation continues.
This commit was SVN r23043.
ompi_ptr_t by translating into void*. Instead keep it as an ompi_ptr_t all
the way. Thanks to Timur Magomedov for helping to track down this issue and
test the patch.
cmr:v1.4
cmr:v1.5
This commit was SVN r23030.
* add hnp and orted modules to the errmgr framework. The HNP module contains much of the code that was in the errmgr base since that code could only be executed by the HNP anyway.
* update the odls to report process states directly into the active errmgr module, thus removing the need to send messages looped back into the odls cmd processor. Let the active errmgr module decide what to do at various states.
* remove the code to track application state progress from the plm_base_launch_support.c code. Update the plm modules to call the errmgr directly when a launch fails.
* update the plm_base_receive.c code to call the errmgr with state updates from remote daemons
* update the routed modules to reflect that process state is updated in the errmgr
* ensure that the orted's open the errmgr and select their appropriate module
* add new pretty-print utilities to print process and job state. Move the pretty-print of time info to a globally-accessible place
* define a global orte_comm function to send messages from orted's to the HNP so that others can overlay the standard RML methods, if desired.
* update the orterun help output to reflect that the "term w/o sync" error message can result from three, not two, scenarios
This commit was SVN r23023.
--enable-mpi-ext configure switch in the top-level README file).
See Josh's excellent wiki page about OMPI extensions:
https://svn.open-mpi.org/trac/ompi/wiki/MPIExtensions
This extension exposes a new API to MPI applications:
{{{
int OMPI_Affinity_str(char ompi_bound[OMPI_AFFINITY_STRING_MAX],
char current_binding[OMPI_AFFINITY_STRING_MAX],
char exists[OMPI_AFFINITY_STRING_MAX]);
}}}
It returns 3 things. Each are a prettyprint string describing sets of
processors in terms of sockets and cores:
1. What Open MPI bound this process to. If Open MPI didn't bind this
process, the prettyprint string says so.
1. What this process is currently bound to. If the process is
unbound, the prettyprint string says so. This string is a
separate OUT parameter to detect the case where some other entity
bound the process (potentially after Open MPI bound it).
1. What processors are availabile in the system, mainly for reference.
This commit was SVN r23018.
and opal_atomic_lifo_pop. Adds memory barriers to remove the race
condition
This commit was SVN r23014.
The following Trac tickets were found above:
Ticket 2355 --> https://svn.open-mpi.org/trac/ompi/ticket/2355
done this way a long time ago for the "gee whiz!" factor -- when in
reality, they really only need one-of-many-run-time priority
selection).
Changed run-time priorities to be as follows:
* darwin: 20
* linux: 20
* posix: 10
* solaris: 30
* test: 5
* windows: 20
I have a very dim (possibly untrue) recollection that Solaris needs to
have a higher priority than others just to ensure that no other is
chosen under Solaris. Make all other "native" components have a
priority of 20 (they shouldn't conflict with each other). Make the
posix fallback component have a priority of 10. Make the test
component priority 5, meaning someone can always select it, but you
can also make a "never select me" component that prioritizes itself
under test.
This commit was SVN r22997.
