Do not pass param file MCA params if the user has requested that no param files be read - required when trying to avoid launch time penalties from large numbers of processes reading default param files. The daemon picks them up and passes them along anyway, so it isn't clear what value we gain from having them all read the defaults
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
This PR renames the common library for OFI libfabric from
libfabric to ofi. There are a number of reasons this
is good to do:
1) its shorter and replaces 9 characters with three for
function names for what may eventually be a fairly extensive interface
2) OFI is the term used for MTL and RML components that use
the OFI libfabric interface
3) A planned OSC component will also use the OFI term.
4) Other HPC libraries that can use OFI libfabric tend to use
the term "ofi" internally and also in their configure options
relevant to OFI libfabric (i.e. MPICH/CH4, Intel MPI, Sandia SHMEM)
There seem to be comments in places in the Open MPI source
code that indicate that this common library will be going away.
Far from it as we will want to be able to share things like
AV objects between OMPI and possibly OSHMEM components that
use the OFI libfabric interface.
This PR also adds a synonym to the --with-libfabric(-libdir)
configury options: --with-ofi and with-ofi-libdir.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
This commit recategorizes several mpirun arguments,
and moves the information for mpirun --help arguments
to the bottom of the general help message. I also
added the OPAL_CMD_LINE_OTYPE field to two commands
that were missed initially because they were not
in the same area as the others.
Signed-off-by: Nathaniel Graham <ngraham@lanl.gov>
The fact that application proc called Abort (read failed) doesn't
mean that ORTE subsystem has failed - vice versa it does it's work
to gracefuly exit the whole application.
orted exiting with non-zero status creates a problem for at least
plm/slurm environments where orteds are launched via `srun` with
"--kill-on-bad-exit" flag. If one of orteds has exited with non-
zero status slurm will immediately kill all other orteds. As the
result we see a lot of leftover in the `/tmp` directory.
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
This commit adds new timing feature that uses environment variables to
expose timing information. This allows easy access to this data (if
timing is enabled) from any other part of the application for the subsequent
postprocessing.
In particular this will be integrated with OMPI-level timing framework that
whill use MPI_Reduce functionality to provide more compact and easy-to use
information.
This commit also adds the example of usage of this framework by annotating
rte_init function. The result is not used anywhere for now. It will be
postprocessed in subsequent commits.
NOTE: that functionality is currently disabled untill it will be verified at runtime
Signed-off-by: Artem Polyakov <artpol84@gmail.com>
Output a (hopefully) helpful message when we timeout an operation
Thanks to Nathan for tracking it down.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
This commit modifies the output from the mpirun --help
command. The options have been split into groups, to
make the output smaller and more readable. The groups
are: general, debug, output, input, mapping, ranking,
binding, devel, compatibility, launch, dvm, and
unsupported. There is also a special "full" command
that can be used to get the old behaviour of printing
out all of the options. Unsupported options may only
be seen with this full output.
This commit also adds a special case for the help
argument. It makes it possible for the user to
enter 0 or 1 arguments instead of having to always
enter an argument. This defaults to printing out
the "general" help options so the user can then
see what help arguments there are.
Signed-off-by: Nathaniel Graham <ngraham@lanl.gov>
Set the daemons' state to "running" and mark them as "alive" by default when constructing the nidmap
Get the DVM running again
Fix direct modex by eliminating race condition caused by releasing data while sending it
Up the size limit before compressing
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
Two updates:
1. Remove the "run as root" error message from orterun.c, because that
functionality is now in orted_submit.c (although it is still
required in orte-dvm.c -- so sync the message in orted_submit.c and
orte-dvm.c to be identical).
2. Slightly tweak the text of the "run as root" error message to
explicitly state that we (strongly) suggest running as a non-root
user (and add a little whitespace).
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
Remove loadleveler as it is obsolescent and is no longer supported.
Fixes#3167
We'll wait for final check of whether or not loadleveler even
compiles/functions before merging this.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
onto the backend daemons. By default, let mpirun only pack the app_context
info and send that to the backend daemons where the mapping will
be done. This significantly reduces the computational time on mpirun as it isn't
running up/down the topology tree computing thousands of binding
locations, and it reduces the launch message to a very small number of
bytes.
When running -novm, fall back to the old way of doing things
where mpirun computes the entire map and binding, and then sends
the full info to the backend daemon.
Add a new cmd line option/mca param --fwd-mpirun-port that allows
mpirun to dynamically select a port, but then passes that back to
all the other daemons so they will use that port as a static port
for their own wireup. In this mode, we no longer "phone home" directly
to mpirun, but instead use the static port to wireup at daemon
start. We then use the routing tree to rollup the initial
launch report, and limit the number of open sockets on mpirun's node.
Update ras simulator to track the new nidmap code
Cleanup some bugs in the nidmap regex code, and enhance the error message for not enough slots to include the host on which the problem is found.
Update gadget platform file
Initialize the range count when starting a new range
Fix the no-np case in managed allocation
Ensure DVM node usage gets cleaned up after each job
Update scaling.pl script to use --fwd-mpirun-port. Pre-connect the daemon to its parent during launch while we are otherwise waiting for the daemon's children to send their "phone home" rollup messages
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
Per a prior commit, the presence of "hwloc.h" can cause ambiguity when
using --with-hwloc=external (i.e., whether to include
opal/mca/hwloc/hwloc.h or whether to include the system-installed
hwloc.h).
This commit:
1. Renames opal/mca/hwloc/hwloc.h to hwloc-internal.h.
2. Adds opal/mca/hwloc/autogen.options to tell autogen.pl to expect to
find hwloc-internal.h (instead of hwloc.h) in opal/mca/hwloc.
3. s@opal/mca/hwloc/hwloc.h@opal/mca/hwloc/hwloc-internal.h@g in the
rest of the code base.
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
This fixes a mismatch between PS listing that returned
USERNAME but code was pruning based on UID.
This changes the OPAL_PS_FLAVOR_CHECK format to return
'uid' instead of 'user'. (Note: Avoiding call to
getlogin_r() but assuming UID is uniform on system,
same assumption exists for session dir anyway.)
Note, still maintains behavior from man page for root
running orte-clean on node (kills all orteds).
Adds 'orte-dvm' to list of procnames that will be checked/killed.
Signed-off-by: Thomas Naughton <naughtont@ornl.gov>
Add ability to pass DVM URI purely via environment
to simplify invocation from command-line (e.g., start dvm,
export URI, mpirun w/o needing to add `--hnp` arg).
If user passes both envvar *and* cmdline, the cmdline wins.
Signed-off-by: Thomas Naughton <naughtont@ornl.gov>
Ensure that job errors do not cause the DVM to fail unless the failed job is the DVM itself.
Refs #2987, with improvements from Ralph
Signed-off-by: Thomas Naughton <naughtont@ornl.gov>
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
Need to check the entire value instead of just the last N digits. Otherwise, "-host 15" will match nid0015, nid0115, and any other launch id ending in 15
It appears strtol can return either a NULL or a zero-length string, so check for both cases
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
This update should fix the mpirun man page so all
mpirun command line options are included, and
mpirun commands that have been removed are no
longer in the man page. I also fixed some of
the file formatting, and bolding of command
parameters.
Signed-off-by: Nathaniel Graham <ngraham@lanl.gov>
Allow someone to specify the "pe=N" modifier to a mapping policy when N=1. This equates to just "bind-to core", but helps people who use a script to set the PE policy. Fix a bug where setting the binding policy left a lingering "if-supported" flag that shouldn't be there.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
Set the orte_bound_at_launch MCA variable. This resolves a launch
performance bug when using aprun to launch Open MPI processes. If
this variable is not set it can take minutes longer to launch with
high ppn.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
* Similar to the other launchers (i.e., slurm, alps) we need to put the
children in a separate process group to prevent SIGINT (from a CTRL-C)
from being delivered to the whole process group and prematurely
killing the rsh/ssh connections to the remote daemons.
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
Fix the check for rsh/ssh so we allow the check for SGE and LoadLeveler to occur if user doesn't specify their own launch agent. Fix a Coverity warning
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
