1
1
Граф коммитов

1439 Коммитов

Автор SHA1 Сообщение Дата
Jeff Squyres
6d026b86b7 Fix a problem reported on the user list by Teng Lin: OPAL_PREFIX
wasn't exported in the Bourne-shell-flavor case on remote nodes.

This commit was SVN r19770.
2008-10-18 12:13:10 +00:00
Jeff Squyres
d96003fec5 Fix typo.
This commit was SVN r19769.
2008-10-18 11:52:41 +00:00
Jeff Squyres
8ea27c0ced Add a missing header file to the Makefile.am so that it can be
included in the distribution tarball.

This commit was SVN r19768.
2008-10-18 11:09:57 +00:00
Ralph Castain
6e5d844c36 Roll in the revamped IOF subsystem. Per the devel mailing list email, this is a complete rewrite of the iof framework designed to simplify the code for maintainability, and to support features we had planned to do, but were too difficult to implement in the old code. Specifically, the new code:
1. completely and cleanly separates responsibilities between the HNP, orted, and tool components.

2. removes all wireup messaging during launch and shutdown.

3. maintains flow control for stdin to avoid large-scale consumption of memory by orteds when large input files are forwarded. This is done using an xon/xoff protocol.

4. enables specification of stdin recipients on the mpirun cmd line. Allowed options include rank, "all", or "none". Default is rank 0.

5. creates a new MPI_Info key "ompi_stdin_target" that supports the above options for child jobs. Default is "none".

6. adds a new tool "orte-iof" that can connect to a running mpirun and display the output. Cmd line options allow selection of any combination of stdout, stderr, and stddiag. Default is stdout.

7. adds a new mpirun and orte-iof cmd line option "tag-output" that will tag each line of output with process name and stream ident. For example, "[1,0]<stdout>this is output"

This is not intended for the 1.3 release as it is a major change requiring considerable soak time.

This commit was SVN r19767.
2008-10-18 00:00:49 +00:00
Jeff Squyres
e34c93c46a Fix problem of missing ) noted by Mostyn Lewis.
This commit was SVN r19758.
2008-10-17 16:03:17 +00:00
Josh Hursey
88aa45dd52 Commit to bring online OpenIB, MX, and shared memory support for Open MPI's checkpoint/restart functionality. Some tuning is still needed, but basic functionality is in place.
There is still a problem with OpenIB and threads (external to C/R functionality). It has been reported in Ticket #1539

Additionally:
* Fix a file cleanup bug in CRS Base.
* Fix a possible deadlock in the TCP ft_event function
* Add a mca_base_param_deregister() function to MCA base
* Add whole process checkpoint timers
* Add support for BTL: OpenIB, MX,  Shared Memory
* Add support Mpool: rdma, sm
* Sundry bounds checking an cleanup in some scattered functions

This commit was SVN r19756.
2008-10-16 15:09:00 +00:00
Ralph Castain
48c3de1865 Fix a problem in the plm "failed to start" code observed by Jeff. When we are unable to launch to a specific node because it doesn't exist or is down, the system would hang and/or segv. The reason for the hang was that we were "firing" the orted exit trigger prior to its timer event being defined - thus "locking" that one-shot and preventing it from firing when we actually were ready to use it.
The segv was caused by the fact that we don't really know which daemon failed to start (at least, in most cases), so we didn't set a pointer to the aborted proc object. All we really wanted, though, was to ensure that mpirun returned a non-zero exit status, so the fix was to simply return the default error status.

This commit was SVN r19754.
2008-10-16 14:21:37 +00:00
Shiqing Fan
3d4e89a5cd - Remove the unused code introduced with r19480, which was for serializing tcp events on Windows and not successful.
This commit was SVN r19747.

The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
  r19480
2008-10-15 08:39:30 +00:00
Shiqing Fan
8b60c755c2 - Bring r19742 into trunk.
- Unify the Windows and the others way of handling callbacks. Thanks to George.
- This will let Windows use the same callbacks as Linux does, which works also.

This commit was SVN r19746.

The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
  r19742
2008-10-15 08:14:24 +00:00
Ralph Castain
a7afa869af Bring Jeff's changes over from v1.2 that restores the automatic source of .profile for bash and ksh shells.
This commit was SVN r19709.
2008-10-08 14:21:42 +00:00
Ralph Castain
f4f81c7308 Let the HNP only update the routing tree if necessary. Enable some debug output
This commit was SVN r19676.
2008-10-03 13:41:08 +00:00
Ralph Castain
0cc2e724f8 Separate var declaration from use to remove compiler warnings in non-debug builds
This commit was SVN r19675.
2008-10-03 13:40:31 +00:00
Ralph Castain
15c47a2473 Revise the daemon collective system to handle comm_spawn patterns that cross into new nodes that are not direct children on the routing tree of the HNP.
Refers to ticket #1548. Although this appears to fix the problem, the ticket will be held open pending further test prior to transition to the 1.3 branch.

This commit was SVN r19674.
2008-10-02 20:08:27 +00:00
Ralph Castain
aa11e0977c Correct a bug in the bookmarking code that incorrectly looked at #slots instead of #slots_allocated, thus causing slot reductions in hostfiles to be ignored when selecting our starting node.
Fixes trac:1527

This commit was SVN r19656.

The following Trac tickets were found above:
  Ticket 1527 --> https://svn.open-mpi.org/trac/ompi/ticket/1527
2008-09-29 14:09:02 +00:00
Ralph Castain
4f89adae0c Prettify the user level display of allocation and map to make it easier to see and understand
This commit was SVN r19655.
2008-09-28 16:44:09 +00:00
Ralph Castain
508cb45583 Add a little more diagnostic info when we cannot do an rml send
This commit was SVN r19654.
2008-09-28 02:13:49 +00:00
Ralph Castain
edb3d99687 Update SLURM environmental variables used to describe allocation. Retain backwards compatibility to SLURM 1.1 and earlier versions.
This commit was SVN r19647.
2008-09-26 02:38:37 +00:00
Kenneth Matney
91bbc6b919 Change algorithm from spawning a shell that spawns another shell, and
thereby runs apstat twice; and in the process thereof reads the ALPS
appinfo file TWICE; and in addition, experiences a failure sometimes
which causes mpirun to hang.  Change this to a looped read attempt
that breaks on success, thereby avoiding failure (except in the most

This commit was SVN r19642.
2008-09-25 20:44:16 +00:00
Ralph Castain
037231fbcb MOdify the node_rank and local_rank fields to be uint16_t so we can handle more than 256 procs/node. Change the type to a defined one so that any future change can be easily done, if required.
This commit was SVN r19637.
2008-09-25 13:39:08 +00:00
Ralph Castain
e64b79f30f Modify the --display-map and --display-alloc per note on devel list to reduce info for user understanding.
Add --display-devel-map and --display-devel-alloc to display all the detailed info we used to provide - it is only of use/interest to developers anyway and confuses users.

This commit was SVN r19608.
2008-09-23 15:46:34 +00:00
Josh Hursey
0cd65bfaa8 Fix a SIGPIPE that may occur when checkpointing a restarted process. This was a result of calling system() in the BLCR CRS. After inspection and testing it was determined that the operation was no longer necessary. So the call was removed thus fixing the bug.
This commit was SVN r19601.
2008-09-22 16:49:56 +00:00
Ralph Castain
16e4b0b698 Ensure that a child job inherits its parent job's prefix dir during comm_spawn operations
This commit was SVN r19538.
2008-09-10 19:05:23 +00:00
Ralph Castain
f326ee356e Add some error output to the plm rsh
This commit was SVN r19532.
2008-09-10 01:59:49 +00:00
Ralph Castain
c0d7fbaf88 A few mapping cleanups - mostly aimed to properly balancing loads so multi app-context comm_spawns don't dump everything on one node.
This commit was SVN r19519.
2008-09-08 15:45:55 +00:00
Ralph Castain
9b8473fdbf Cleanup orted cmd line - we don't need to pass nodenames, and shouldn't pass heartbeat unless the orted is going to use it. This helps shorten the cmd line for future use.
Cleanup when an orted actually opens the PLM. Unfortunately, some unmentionable people are pushing head node environs out to remote nodes, causing the daemons to think they are the HNP. This helps prevent the confusion.

This commit was SVN r19518.
2008-09-08 15:45:11 +00:00
Shiqing Fan
04ee20a880 - Mainly type casts. Microsoft VC++ compiler is too strict.
This commit was SVN r19517.
2008-09-08 15:39:30 +00:00
Josh Hursey
edf52e7258 This commit should fix and close #1482
The problem was that (outside of Odin configure issues) that the IOF is no longer enabled by application processes.

Checkpoint/restart seems to be working once again.

Thanks to Ralph for pointing me here.

This commit was SVN r19508.
2008-09-05 18:39:17 +00:00
George Bosilca
579d70edad We should use #ifdef and not #if
This commit was SVN r19504.
2008-09-05 12:44:19 +00:00
Brian Barrett
79cf946bce Add header file needed for the non-full RTE, non-debug case
This commit was SVN r19475.
2008-09-01 18:02:32 +00:00
Shiqing Fan
cd6ff74d89 Update the ccp module:
rename the get_cluster_message function for both ras/plm.
  use _umask instead of umask.
  add WIN32_DCOM definition to support Windows Vista.

This commit was SVN r19470.
2008-09-01 16:35:38 +00:00
Brian Barrett
a15171e46b Some fixes for the disabled ORTE case
* Protect an orte variable used in the orte debugger stuff
  * Initialize the datatype code in the Catamount code, as we need it
    for intercommunicators (the proc code needs it to pack the remote
    name)
  * Turn on a bunch of the orte datatype code so that ORTE_NAME is available.

This commit was SVN r19457.
2008-08-31 18:06:55 +00:00
Shiqing Fan
94a2147e3d - make sure that the system has the header files.
This commit was SVN r19400.
2008-08-25 13:56:10 +00:00
Ralph Castain
b45029fd0e Application processes should not open/close the IOF framework - there is nothing in that framework for application procs to do.
Fix a bug in iof_base_close where we destruct a thread lock prior to unlocking it.

This commit was SVN r19392.
2008-08-22 01:28:19 +00:00
Ralph Castain
4ef9d15d97 Revamp the opal mca paffinity interface. We ran into a problem when we encountered machines that had "holes" in their physical processor layout - e.g., machines that supported "hotplugging", or that had unpopulated sockets. To solve that problem, we had to clarify at the API level where we were describing physical vs logical processor info, and then translate accordingly in the underlying implementation.
See opal/mca/paffinity/paffinity.h for explanation as to the physical vs logical nature of the params used in the API.

Fixes trac:1435

This commit was SVN r19391.

The following Trac tickets were found above:
  Ticket 1435 --> https://svn.open-mpi.org/trac/ompi/ticket/1435
2008-08-21 19:21:28 +00:00
Ralph Castain
43f8bcfe54 Update slurm plm to respect leave_session_attached
This commit was SVN r19370.
2008-08-19 18:30:30 +00:00
Ralph Castain
4e0f34a062 When we hit an error prior to actually launching daemons, it would be nice if orterun didn't bark about daemons failing to launch, mpirun detecting a job failed, etc.
Add a new job state to indicate that we never attempted to launch. Flag such a scenario and avoid hitting all the other error messages.

This commit was SVN r19366.
2008-08-19 15:19:30 +00:00
Ralph Castain
9447334749 Some comments relating to relative indexing
This commit was SVN r19365.
2008-08-19 15:17:40 +00:00
Ralph Castain
49745c5f40 Provide a new option that allows a user to leave an ssh session open without getting deluged by ORTE debug output. The new option is --leave-session-attached, with a corresponding MCA param of orte_leave_session_attached.
Theoretically, any PLM could use this - but in reality, all of them except rsh/ssh already leave the session attached anyway.

This fixes trac:656 - a REALLY old ticket

This commit was SVN r19294.

The following Trac tickets were found above:
  Ticket 656 --> https://svn.open-mpi.org/trac/ompi/ticket/656
2008-08-14 18:59:01 +00:00
Ralph Castain
dd16e4e4a6 Update process component of odls.
This commit was SVN r19281.
2008-08-13 20:07:59 +00:00
Ralph Castain
913cf04633 Only co-locate debugger daemon if the orted has local children - prevents mpirun from co-locating a daemon when it has no local procs
This commit was SVN r19280.
2008-08-13 20:06:28 +00:00
Ralph Castain
3e2a3db887 Add a missing ntoh conversion when pushing a message back onto the RML progress queue.
If a message cannot be routed because the addressee isn't yet known, then the message is held on a queue in the RML for a period of time (currently set to 500 millisec). At the end of that time, we pop the message from the list and attempt to send it again. This action requires that we convert the header back to
network-byte-order before calling the OOB.

If the message still cannot be routed, we put the message back on the list and reset the timer. However, since we are going to convert the header when it com
es off of the list, we have to ntoh it before putting it back on the list so it all comes out right. This step was missing.

Thus, the problem only showed up relatively rarely because a message would have to be pushed onto the queue at least twice for the problem to surface.

This should fix a specific ticket (1389), but we will wait to see the results of MTT runs to verify. Note that we really don't know why a message is rattling around in the RML for so long, especially since this all seems to be happening during finalize, so this could cause mpirun to hang. Or it could simply trash the message and exit cleanly. Shall be interesting to see!

This commit was SVN r19276.
2008-08-13 17:54:15 +00:00
Ralph Castain
30f37f762d Enable co-location of debugger daemons during initial launch and when debugging a running job.
Provide support for four MPIR extensions that allow specification of debugger daemon executable, argv for the debugger daemon, whether or not to forward debugger daemon IO, and whether or not debugger daemon will piggy-back on ORTE OOB network. Last is not yet implemented.

No change in behavior or operation occurs unless (a) the debugger specifically utilizes the extensions and, for co-locate while running, the user specifically enables the capability via an MCA param. Two of the MPIR extensions supported here are used in a widely-used debugger for a large-scale installation. The other two extensions are new and being utilized in prototype work by several debuggers for possible future release.

This commit was SVN r19275.
2008-08-13 17:47:24 +00:00
Rainer Keller
d57ef70149 - Store the result of the 1-byte read... and assert, in case
of error checking -- we don't return errors here anyway.
   Fixes Coverity CID 981

This commit was SVN r19259.
2008-08-12 18:00:38 +00:00
Ralph Castain
f017c55bfa Close a minor memory leak - we can reuse timer events
This commit was SVN r19251.
2008-08-12 12:53:30 +00:00
Ralph Castain
baed5dcad0 Ensure contact info is placed in the job family session directory so orte-ps and other tools can find it
This commit was SVN r19245.
2008-08-11 23:48:39 +00:00
Rainer Keller
0f8a80d81d - The intel compiler does not play nice with the
__opal_attribute_format__ on typedef defined functions and
   emits a warning once errmgr.h is included --> read: often...

This commit was SVN r19229.
2008-08-08 16:26:09 +00:00
Jeff Squyres
797ec531aa Some more work on the man pages:
* Make the creation of the build dir for the man pages a bit more
   robust (thanks to suggestions from Ralf W.).
 * Only distribute the .Xin files, not the .X man pages themselves.
 * Make the .X files depend on opal_config.h so that if you re-run
   configure and change opal_config.h (e.g., a new version), the man
   pages should get rebuilt.
 * Man pages are now cleaned with "distclean", not "maintainer-clean".
 * Fix a typo in opal_crs.7in.
 * Udpate make_dist_tarball to update "date" in the VERSION file.
 * Make make_dist_tarball a bit friendlier to hg checkouts.

This commit was SVN r19219.
2008-08-07 19:20:40 +00:00
Rainer Keller
ad3538ea38 - Per Discussion w/ Ralph Castain (related to CID 1051)
- Move up the __opal_attribute_noreturn__ information
 - Actually make it known outside in ess.h
 - Additionally allow printf-type checking

This commit was SVN r19210.
2008-08-07 09:36:10 +00:00
Ralph Castain
c9e53fd0d4 Add capability to notify system admins of potential problems in system communication networks and/or other system elements that are detected by Open MPI during operation. For example, failures in connections that may be indicative of connectivity problems can be reported to sys admins in addition to our current error message to the user, thus allowing more rapid correction of the problem.
This system is "off" by default and only operates upon specific directive for selection of a notifier component. At the moment, the only available component will write an error message to the syslog.

This commit was SVN r19209.
2008-08-06 21:59:21 +00:00
Rainer Keller
c58e89e471 - Fix variable set but not used
Coverity CID1061

This commit was SVN r19197.
2008-08-06 14:42:16 +00:00