1
1
Граф коммитов

4291 Коммитов

Автор SHA1 Сообщение Дата
Ralph Castain
4316213805 Fix add-host support by including the location for procs of prior jobs when spawning new daemons.
Thanks to CalugaruVaxile for the report

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-12-07 14:48:58 -08:00
Gilles Gouaillardet
4a481f66e6 odls/base: fix orte_odls_base_harvest_threads()
Do not try to finalize odls progress threads if they have not been started yet

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-12-04 15:18:04 +09:00
Gilles Gouaillardet
3496897961 odls/base: fix handling of the odls_base_num_threads MCA param
If a number of odls threads is explicitly required, then use
that number no matter what.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-12-04 11:19:25 +09:00
Gilles Gouaillardet
a4755b694b odls/pspawn: record the pid of the spawn'ed process
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-12-01 15:00:50 +09:00
Ralph Castain
b5bf0a7f1d Add a new posix_spawn component to the ODLS framework.
Only selectable when specifically requested via "-mca odls pspawn"

Note that there are several concerns:
  * we aren't getting SIGCHLD calls when the procs terminate
  * we aren't seeing the IO pipes close on termination, though
    we are getting output forwarded to mpirun
  * I haven't found a way to bind the child process prior to exec.
    If we want to use this method, we probably need someone to
    implement a cgroup component for the orte/rtc framework

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-30 18:01:31 -08:00
Ralph Castain
335fc96f42 Remove debug
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-29 21:21:35 -08:00
Gilles Gouaillardet
8e17127258 plm/alps: fix orte_wait_cb() usage
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-11-30 13:49:22 +09:00
Ralph Castain
8f496b01b7 Try automatically adding local spawn threads to parallelize the fork/exec process to speed up the launch on large SMPs. Harvest the threads after initial spawn to minimize any impact on running jobs.
Change the determination of #spawn threads to be done on basis of #local procs in first job being spawned. Someone can look at an optimization that handles subsequent dynamic spawns that might be larger in size.

Leave the threads running, but blocked, for the life of the daemon, and use them to harvest the local procs as they terminate. This helps short-lived jobs in particular.

Add MCA params to set:
  * max number of spawn threads (default: 4)
  * set a specific number of spawn threads (default: -1, indicating no set number)
  * cutoff - minimum number of local procs before using spawn threads (default: 32)

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-29 19:54:00 -08:00
Josh Hursey
38aefd2ec4
Merge pull request #4439 from mattaezell/lsf_csm
Disable the LSF plm if CSM is detected
2017-11-29 07:43:16 -06:00
Yu Feng
6aaf62584b Mention --oversubscribe
The current error message when the number of slots is insufficient
(e.g. running mpirun -n 4 on a dual core machine) does not mention the
use of `--oversubscribe`.

In earlier version of Open MPI, the over-subscription was automatic
(albeit buggy?); but the important point was no error message was
printed and the application runs.  Mentioning the oversubscibe flag in
the message will ease up the transition to the current behaviour where
explicit request is required.

Also make a few other minor tweaks / cleanups to the
orte-rmaps-seq:alloc-error help message.

Signed-off-by: Yu Feng <rainwoodman@gmail.com>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2017-11-27 10:14:13 -05:00
Ralph Castain
1de0421e48 Provide a more robust way of checking for proct completion
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-26 10:39:53 -08:00
Ralph Castain
a25a7bcba7 Handle the case where stdout and stderr get merged into a file
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-26 08:18:34 -08:00
Ralph Castain
e3c308dfc8 Update the odls/alps component
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-25 19:51:07 -08:00
Ralph Castain
3906aaf41a Silence warnings
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-25 11:50:18 -08:00
Ralph Castain
30f23ac67a Save one more file descriptor per process by not opening one for stddiag
if PMIx (version > 1.x) is active since all diagnostic messages will instead flow thru
the PMIx connection. Unfortunately, PMIx v1 does not support this
feature, but we can remove the stddiag support once PMIx v1 slides out
of the support window

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-25 11:48:53 -08:00
Gilles Gouaillardet
e88767866e iof: optimize handling of stderr when iof_base_redirect_app_stderr_to_stdout is set
avoid creating a pipe for a task stderr when we know it will be redirected to stdout

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-11-24 13:20:03 +09:00
Gilles Gouaillardet
84e96522f2 iof: optimize handling of stdin
since some tasks migth end up having /dev/null as their stdin,
simply avoid pipe creation and destruction for these tasks.

From a pragmatic and MPI point of view, and unless explicitly required
otherwise, all MPI tasks but (the first) one end up with /dev/null
as their stdin.

Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-11-22 13:18:32 +09:00
Gilles Gouaillardet
47bf0d6f9d iof/base: do not assume fileno(stdin) is zero
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
2017-11-22 11:45:13 +09:00
Boris Karasev
d2a568afa5 rmaps/mindist: reworked the job map binding
The following issues have been fixed for `mindist`:
- computing the job map on the backend nodes
- using slots count (`-host node1:<s1>,nodeN:<sN>`)
- fixed `dist:span` job mapping method
- fixed `oversubcribe` option with `-host`

Signed-off-by: Boris Karasev <karasev.b@gmail.com>
2017-11-09 08:56:44 +02:00
Ralph Castain
dcf389b6fa We now add all nodes to the job data object when we map, so don't do it twice
Fixes #4449

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-11-05 17:17:50 -08:00
Matt Ezell
e45761d498 Disable the LSF plm if CSM is detected
LSF running on top of CSM does not provide LSF daemons on the compute nodes.

Signed-off-by: Matt Ezell <ezellma@ornl.gov>
2017-11-02 13:48:46 -04:00
George Bosilca
8f32b345de
Address syslog issues on OSX 10.13 with gcc 7.x
gcc 7.[1,2] (at least) fails to correctly parse the OSX 10.13 sys/syslog.h
header. As a results we need to potect syslog support in OPAL, PMIX and
ORTE.

Signed-off-by: George Bosilca <bosilca@icl.utk.edu>
Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2017-10-23 14:02:10 -04:00
Ralph Castain
75d411f3ea Ensure we update the routing plan so that tree spawn works!
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-10-19 14:02:06 -07:00
Ralph Castain
388034c814 Add support for the -v (verbose) option to prun and silence the "executing" and "completed" output otherwise.
Debounce "unreachable" notifications for tools when they disconnect
Enable the -x cmd line option for prun

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 0a5b36180a22959654461ac1303cec35313f8b4a)
2017-10-10 12:54:49 -07:00
Ralph Castain
c696e04c5e Since PMIx is moving to release v3.0, embed the new release candidate in opal/pmix framework. Move the pmix2x code over to the ext2x component. Create a new ext3x component
Remove some build product. Tell PMIx that we don't need a new nspace generated when OMPI calls connect
Add missing Makefile

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-10-09 13:51:08 -07:00
Ralph Castain
5352c31914 Enable remote tool connections for the DVM. Fix notifications so we "de-bounce" termination calls
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-10-06 10:47:05 -07:00
Ralph Castain
d7d127b9b5 Correctly assign locales when mapping ppr
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-10-03 03:03:56 -07:00
Ralph Castain
d5ce3c38e1 Begin cleaning up debugger support
Debugger daemons do not count against available slots. Clean up some leftover errors from the upgrade to HWLOC 2 in the mappers. Properly flag debugger jobs that come in via PMIx.

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-27 16:18:43 -05:00
Ralph Castain
fcb7a2f29b Minor cleanups for when using external pmix
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-24 09:53:04 -07:00
Ralph Castain
fe9b584c05 Fully support OMPI spawn options. Fix a bug in the round-robin mappers where we weren't adding nodes to the job map node array, and so resources were not released
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 285d8cfef74ffc899e9c51e1d9c597b7fb2ceb89)
2017-09-21 10:29:27 -07:00
Ralph Castain
e575c4d6f9 Fix tool connection logic so we properly search for default session server, perform specified number of retries, etc.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 7c755e01004f8b86c71f1729662979ea45ab1adb)
2017-09-19 13:35:46 -07:00
Ralph Castain
16de607607 Merge pull request #4234 from rhc54/topic/upstream
Ensure we update the total_slots_alloc field on each job. Correct the client example
2017-09-19 09:03:04 -07:00
Ralph Castain
658c3d1d51 Ensure we update the total_slots_alloc field on each job. Correct the client example
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit bcedd12a8a24dd246f04ff13b4fd2f1bbac6ce5a)
2017-09-19 08:14:14 -07:00
Jeff Squyres
7cccee9d92 rmaps/base: remove debugging "DONE" message
Thanks for Ben Menadue for reporting and supplying the patch.

Signed-off-by: Jeff Squyres <jsquyres@cisco.com>
2017-09-19 07:10:00 -07:00
Ralph Castain
5708872112 Implement support for "local" range when publishing data
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
(cherry picked from commit 2d54f7e0dd3a47260b0b2634aae3361316005933)
2017-09-18 19:34:08 -07:00
Josh Hursey
252be7ffb0 Merge pull request #4215 from jjhursey/fix/plm-lsf-rc
plm/lsf: Improve error message if lsb_launch fails
2017-09-18 11:14:25 -05:00
Ralph Castain
3c914a7a97 Complete the fix of the ORTE DVM. We will now use "prun" instead of "orterun -hnp foo" to execute jobs. This provides the feature of automatic discovery of the orte-dvm so you don't need to manually enter URI's or contact file locations. All IO is forwarded to prun.
Still in the "needs to be done" category:

* mapping/ranking/binding options aren't correctly supported

* if the DVM encounters some errors (e.g., not enough resources for the job), the resulting error is globally set and impacts any subsequent job submission

Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-16 13:13:07 -07:00
Joshua Hursey
89c1aaf646 plm/lsf: Improve error message if lsb_launch fails
Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
2017-09-15 09:45:58 -05:00
Ralph Castain
7c7d8a69a0 Backport changes from PMIx reference server
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-14 11:48:56 -07:00
anandhi
4d7de8882f Checking for generic transport "fabric" in mca parameter rml_ofi_transports
to choose the first available non-socket provider.
	modified:   orte/mca/rml/ofi/rml_ofi_component.c
	modified:   orte/mca/rml/ofi/rml_ofi_send.c

Signed-off-by: Anandhi Jayakumar <anandhi.s.jayakumar@intel.com>
2017-09-12 15:39:55 -07:00
Ralph Castain
3477079804 Repair the ORTE DVM
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-09-11 17:38:21 -07:00
Howard Pritchard
5db9416724 rml/ofi: swat a compiler warning
On the path to -Werror passing builds!

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2017-08-30 09:16:49 -06:00
Josh Hursey
ad87aa2674 Merge pull request #4121 from jjhursey/explore/dlopen-local
mca: Dynamic components link against project lib
2017-08-25 13:15:51 -05:00
Joshua Hursey
e1d079544b mca: Dynamic components link against project lib
* Resolves #3705
 * Components should link against the project level library to better
   support `dlopen` with `RTLD_LOCAL`.
 * Extend the `mca_FRAMEWORK_COMPONENT_la_LIBADD` in the `Makefile.am`
   with the appropriate project level library:
```
MCA components in ompi/
       $(top_builddir)/ompi/lib@OMPI_LIBMPI_NAME@.la
MCA components in orte/
       $(top_builddir)/orte/lib@ORTE_LIB_PREFIX@open-rte.la
MCA components in opal/
       $(top_builddir)/opal/lib@OPAL_LIB_PREFIX@open-pal.la
MCA components in oshmem/
       $(top_builddir)/oshmem/liboshmem.la"
```

Note: The changes in this commit were automated by the script in
the commit that proceeds it with the `libadd_mca_comp_update.py`
script. Some components were not included in this change because
they are statically built only.

Signed-off-by: Joshua Hursey <jhursey@us.ibm.com>
2017-08-24 11:56:16 -04:00
Ralph Castain
68029b27e4 Fix the orte-dvm operations so that orterun can connect and execute an application. There is a lingering problem, though. The first invocation of orterun succeeds every time. However, subsequent invocations have a high probability of hanging in the OOB connection handshake.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-08-23 17:31:08 -07:00
Ralph Castain
d80b0c7990 If the HWLOC shared memory system is unable to connect, then fallback to providing the topology via XML. Do not automatically provide the XML to every process as that defeats the purpose of the shared memory system. Instead, use PMIx_Query_info_nb to get the info from the server when required.
Signed-off-by: Ralph Castain <rhc@open-mpi.org>
2017-08-22 18:12:26 -07:00
Brice Goglin
046d870124 rtc/hwloc/shmem: add Inria copyrights
The code for finding the hole for the shmem region actually came from me.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
2017-08-21 23:09:57 +02:00
Brice Goglin
baf762d99d rtc/hwloc/shmem: dump /proc/self/maps if failed to find a hole and verbosity > 4
Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
2017-08-21 19:57:38 +02:00
Brice Goglin
8f6afbb641 rtc/hwloc/shmem: fix "heap" hole search kind
There can be multiple [heap] consecutively in proc/<pid>/maps,
and there's no room between them.
Don't use a hole after the first [heap] is there's another [heap]
immediately after it.

This code would fail to find the last [heap] if there were multiple
[heap] interleaved with non-heap VMA, but our kind "after heap"
wouldn't be meaningful anymore anyway.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
2017-08-21 15:42:38 +02:00
Brice Goglin
b8b46b253b rtc/hwloc/shmem: fix "libs" hole search kind
We want the biggest hole *between* heap and stack, not outside.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
2017-08-21 15:40:36 +02:00