ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность
17 834 коммитов 3 Ветки 0 Теги
Граф коммитов

381 Коммитов

Автор SHA1 Сообщение Дата
Ralph Castain
e4673f3283 Add new job state 
This commit was SVN r27878.
 2013-01-20 00:30:27 +00:00
Ralph Castain
ab73d11368 Oops - push missing definitions 
This commit was SVN r27688.
 2012-12-18 16:43:03 +00:00
Ralph Castain
43f883cb42 Add some more detailed error output to the db_hash component and nidmap code. Ensure the local nodename is included in the HNP's aliases 
This commit was SVN r27622.
 2012-11-18 17:57:19 +00:00
Ralph Castain
e11f32038a Add an MCA param to retain all aliases based on IP addrs for node names so that procs can look them up by interface, if desired. If the param is set, pass aliases around to all daemons and procs for local use 
This commit was SVN r27619.
 2012-11-16 04:04:29 +00:00
Ralph Castain
bd887f7f56 Add a new "test" component to the DFS that treats all files as remote in order to test the app-to-daemon interactions on a single machine. Set a global param to indicate we are using staged execution. Add a param to indicate it is okay for non-MPI processes to execute without finalizing. Cleanup file map load and fetch operations. 
This commit was SVN r27587.
 2012-11-10 14:09:12 +00:00
Ralph Castain
81d0b06842 Strip the domain info from the hostname if that option is specified, protecting IP address-based names 
This commit was SVN r27586.
 2012-11-10 14:05:27 +00:00
Nathan Hjelm
842caae4c7 Fix a small leak in orte/util/name_fns.c 
cmr:v1.7

This commit was SVN r27576.
 2012-11-07 23:59:49 +00:00
Ralph Castain
27b41a7db4 If the nodename is an IP address, we need to retain the full name (even if keep_fqdn is false) so that the ssh tree spawn can proceed. 
cmr:v1.7

This commit was SVN r27561.
 2012-11-05 16:59:53 +00:00
Nathan Hjelm
2acd0f83de Revert "Revert r27451 and r27456 - the cmd line parser is incorrectly marking the application as an MCA parameter". 
It appears the problem was not with the command line parser but the rsh plm. I don't know why this problem was not occuring before the command line parser changes but it appears to be resolved now.

This commit was SVN r27527.

The following SVN revision numbers were found above:
  r27451 --> open-mpi/ompi@d59034e6ef
  r27456 --> open-mpi/ompi@ecdbf34937
 2012-10-30 19:45:18 +00:00
Ralph Castain
a080de188f Enable orterun to directly support staged execution, treating each app as a separate job. Support transfer of file maps when support exists. 
This commit was SVN r27516.
 2012-10-29 23:11:30 +00:00
Ralph Castain
e6014bf2e1 Revert r27451 and r27456 - the cmd line parser is incorrectly marking the application as an MCA parameter 
This commit was SVN r27477.

The following SVN revision numbers were found above:
  r27451 --> open-mpi/ompi@d59034e6ef
  r27456 --> open-mpi/ompi@ecdbf34937
 2012-10-24 18:38:44 +00:00
Nathan Hjelm
d59034e6ef MCA: remove deprecated mca_base_param functions (mca_base_param_register_int, mca_base_param_register_string, mca_base_param_environ_variable). Remove all uses of deprecated functions. 
cmr:v1.7

This commit was SVN r27451.
 2012-10-17 20:17:37 +00:00
Ralph Castain
54db4c35eb Get the trunk to build again when --without-hwloc is specified. Move a couple of key type definitions and utilities out from under the HAVE_HWLOC test so they are always available as they don't really depend on hwloc's presence. Tell two compnents not to build if hwloc is disabled: 
ompi/mca/sbgp/basesmsocket
orte/mca/rmaps/lama

Remove stale configure.params files from the sbgp framework as the OMPI build system no longer looks at those files.

This commit was SVN r27377.
 2012-09-26 23:24:27 +00:00
Ralph Castain
67f34c3be6 Record the bind_level recvd by the daemon for each job so it can be correctly sent to the procs. Add test in get_relative_locality to avoid descending into an infinite loop if the level is NODE (==0). 
This commit was SVN r27252.
 2012-09-06 20:50:07 +00:00
Ralph Castain
efa50346c8 Error out if we are filtering a hostfile and encounter a node that is not in the resource-managed allocation, giving an error message identifying the file and the node. Don't filter managed allocations thru a default hostfile as this can lead to "hidden" errors. 
Don't use dash-host info on managed allocations if we using soft locations

This commit was SVN r27245.
 2012-09-05 19:42:00 +00:00
Ralph Castain
fde83a44ab This confusion has been around for awhile, caused by a long-ago decision to track slots allocated to a specific job as opposed to allocated to the overall mpirun instance. We eliminated that quite a while ago, but never consolidated the "slots_alloc" and "slots" fields in orte_node_t. As a result, confusion has grown in the code base as to which field to look at and/or update. 
So (finally) consolidate these two fields into one "slots" field. Add a field in orte_job_t to indicate when all the procs for a job will be launched together, so that staged operations can know when MPI operations are allowed.

This commit was SVN r27239.
 2012-09-05 01:30:39 +00:00
Ralph Castain
bae5dab916 If (and only if) a user requests, set the default number of slots on any node to the number of objects of the specified type. This *only* takes effect in an unmanaged environment - i.e., if an external resource manager assigns us a number of slots, then that is what we use. However, if we are using a hostfile, then the user may or may not have given us a value for the number of slots on each node. 
For those nodes (and *only* those nodes) where the user does *not* specify a slot count, we will set the number of slots according to their direction: either to the number of cores, numas, sockets, or hwthreads. Otherwise, the slot count is set to 1.

Note that the default behavior remains unchanged: in the absence of any value for #slots, and in the absence of any directive to set #slots, we will set #slots=1.

This commit was SVN r27236.
 2012-09-04 20:58:26 +00:00
Ralph Castain
95019cc310 Fix a few places where we weren't completely identifying hostfile-based operations against "localhost" entries. Tell the mapper base to be silent when we don't want errors announced because nodes aren't available for mapping (something it is okay if they are fully used). Fix an infinite loop in the file prepositioning code. 
This commit was SVN r27210.
 2012-08-31 21:28:49 +00:00
Ralph Castain
38ce23db43 Add some protection to allow NULL bytes in byte objects and NULL strings to be handled cleanly in nidmaps and modex entries. Ensure there is a valid nidmap available for the HNP to pass down to any local procs when it is operating alone. 
This commit was SVN r27188.
 2012-08-31 01:07:36 +00:00
Ralph Castain
05c0464dcb Add missing protections 
This commit was SVN r27183.
 2012-08-30 12:17:29 +00:00
Ralph Castain
1b659de132 Get staged execution working on multi-node setups. Improve efficiency by only remapping if all procs not yet mapped in the job. 
This commit was SVN r27181.
 2012-08-29 20:35:52 +00:00
Ralph Castain
a3b08f5800 Fix a few things relating to comm_spawn that causes new daemons to be launched. Ensure that all new daemons receive a full pidmap. Properly mark the daemon job as "updated" when daemons are added 
This commit was SVN r27177.
 2012-08-29 03:11:37 +00:00
Ralph Castain
98580c117b Introduce staged execution. If you don't have adequate resources to run everything without oversubscribing, don't want to oversubscribe, and aren't using MPI, then staged execution lets you (a) run as many procs as there are available resources, and (b) start additional procs as others complete and free up resources. Adds a new mapper as well as a new state machine. 
Remove some stale configure.m4's we no longer need.

Optimize the nidmaps a bit by only sending info that has changed each time, instead of sending a complete copy of everything. Makes no difference for the typical MPI job - only impacts things like staged execution where we are sending multiple (possibly many) launch messages.

This commit was SVN r27165.
 2012-08-28 21:20:17 +00:00
Ralph Castain
229e3f9b2a This will break systems like orcm, but we aren't trying to support those any more - so put the nodes back in their daemon-indexed position. Will continue working to reduce search requirements in other parts of the code 
This commit was SVN r27038.
 2012-08-14 22:26:40 +00:00
Ralph Castain
3cb8d55c8b We can't just lookup the node in the node pool by daemon vpid as the daemons aren't stored that way - this was done because when holes exist in daemon vpids, we can generate huge orte_node_pool arrays even when only a few daemons actually exist. So we have to search for the vpid in the array 
This commit was SVN r27035.
 2012-08-14 18:17:59 +00:00
Ralph Castain
3938ec5361 Remove debug 
This commit was SVN r27024.
 2012-08-13 21:35:21 +00:00
Ralph Castain
b9b41d8662 For cases where the alpha+non-zero prefix must be removed from a node name, be sure to do it everywhere we access node names - otherwise, modex methods such as pmi will fail to correctly identify procs on the same node 
This commit was SVN r27022.
 2012-08-13 20:44:56 +00:00
Ralph Castain
431d5361ed For those who really preferred our prior mode of operation that mapped procs and only launched daemons on the nodes that had procs on them, introduce the "novm" state machine component. This recreates the old mode of operation by re-ordering the launch sequence so that we allocate, then map, and then launch daemons only on the reqd nodes (instead of across the entire allocation). 
This commit was SVN r26946.
 2012-08-03 16:30:05 +00:00
Ralph Castain
6ee35e4977 Add num_local_peers to orte_process_info so we don't keep re-computing it, ensure it is available for direct launch via pmi as well 
This commit was SVN r26931.
 2012-07-31 21:21:50 +00:00
Ralph Castain
94d11e04fd Add an intermediate state when the VM is ready so that third party tools can take action prior to mapping/launching apps 
This commit was SVN r26902.
 2012-07-28 15:33:09 +00:00
Ralph Castain
cf4606cdd5 Add debug of nidmap subsystem 
This commit was SVN r26739.
 2012-07-04 00:04:16 +00:00
Ralph Castain
b83fc41d54 Add a state that allows mpirun or other tools to be notified of a job completion prior to terminating so that alternative actions can be performed. 
This commit was SVN r26716.
 2012-07-02 22:16:32 +00:00
Ralph Castain
0dfe29b1a6 Roll in the rest of the modex change. Eliminate all non-modex API access of RTE info from the MPI layer - in some cases, the info was already present (either in the ompi_proc_t or in the orte_process_info struct) and no call was necessary. This removes all calls to orte_ess from the MPI layer. Calls to orte_grpcomm remain required. 
Update all the orte ess components to remove their associated APIs for retrieving proc data. Update the grpcomm API to reflect transfer of set/get modex info to the db framework.

Note that this doesn't recreate the old GPR. This is strictly a local db storage that may (at some point) obtain any missing data from the local daemon as part of an async methodology. The framework allows us to experiment with such methods without perturbing the default one.

This commit was SVN r26678.
 2012-06-27 14:53:55 +00:00
Josh Hursey
28681deffa Backout the ORCA commit. :( 
There is a linking issue on Mac OSX that needs to be addressed before this is able to come back into the trunk.

This commit was SVN r26676.
 2012-06-27 01:28:28 +00:00
Josh Hursey
542330e3a7 Commit of ORCA: Open MPI Runtime Collaborative Abstraction 
This is a runtime interposition project that sits between the OMPI and ORTE layers in Open MPI.

The project is described on the wiki:
  https://svn.open-mpi.org/trac/ompi/wiki/Runtime_Interposition

And on this email thread:
  http://www.open-mpi.org/community/lists/devel/2012/06/11109.php

This commit was SVN r26670.
 2012-06-26 21:42:16 +00:00
Ralph Castain
0a713cd27e Add database framework to ORTE and refactor modex code to utilize it. Create the "hash" db component from the prior modex db code. Leave the other components ignored for now - will activate them later. 
Modex is still a blocking operation at this point.

This commit was SVN r26618.
 2012-06-19 13:38:42 +00:00
Ralph Castain
9bedb25dda Cleanup some compiler warnings, some of which are actual logic errors 
This commit was SVN r26519.
 2012-05-29 20:11:51 +00:00
Ralph Castain
e705de1ce6 Complete nidmap cleanup - we don't know our node until we have unpacked all the jobs since our job is always the last one, so wait until all jobs are unpacked before assigning locality 
This commit was SVN r26500.
 2012-05-27 18:37:57 +00:00
Ralph Castain
be6ed9c2df Allow partial use of allocations by specifying the max number of daemons (i.e., max VM size) for the job 
This commit was SVN r26499.
 2012-05-27 16:48:19 +00:00
Ralph Castain
c69a04e16b Cleanup the pidmap decoding for apps to avoid confusion 
This commit was SVN r26498.
 2012-05-27 16:21:38 +00:00
Nathan Hjelm
6eeca66475 add an option to enable static ports. diabled by default 
This commit was SVN r26462.
 2012-05-21 19:56:15 +00:00
Ralph Castain
84d031d6c1 Add daemon object to job array after creation 
This commit was SVN r26406.
 2012-05-08 13:39:20 +00:00
Ralph Castain
b2f77bf08f Extend the iof by adding two new components to support map-reduce IO chaining. Add a mapreduce tool for running such applications. 
Fix the state machine to support multiple jobs being simultaneously launched as this is not only required for mapreduce, but can happen under comm-spawn applications as well.

This commit was SVN r26380.
 2012-05-02 21:00:22 +00:00
Ralph Castain
a8db2fc95f Add procs to each node's map on the daemons 
This commit was SVN r26368.
 2012-05-01 16:41:35 +00:00
Ralph Castain
289f9f41ec From long-term discussions, have the daemons use the node_t and proc_t structs and arrays instead of the pidmap and nidmap arrays. Sets the stage for future work. 
This commit was SVN r26359.
 2012-04-29 00:10:01 +00:00
Ralph Castain
3461809341 Fix reporting of launch progress so the numbers are correct and appear when they should 
This commit was SVN r26342.
 2012-04-26 00:10:09 +00:00
Ralph Castain
9cd4c06488 Get things to build and run when --disable-orte is specified 
This commit was SVN r26263.
 2012-04-10 21:50:01 +00:00
Ralph Castain
bd8b4f7f1e Sorry for mid-day commit, but I had promised on the call to do this upon my return. 
Roll in the ORTE state machine. Remove last traces of opal_sos. Remove UTK epoch code.

Please see the various emails about the state machine change for details. I'll send something out later with more info on the new arch.

This commit was SVN r26242.
 2012-04-06 14:23:13 +00:00
Josh Hursey
1941f6b3b1 Cleanup some compiler warnings when doing an optimized/non-debug build. 
This commit was SVN r26236.
 2012-04-04 20:40:16 +00:00
Ralph Castain
75f738bfce Fix one last place 
This commit was SVN r26092.
 2012-03-03 00:39:37 +00:00