1
1
Граф коммитов

198 Коммитов

Автор SHA1 Сообщение Дата
Pak Lui
08352878cc * Added in new ras and pls components to support Sun N1 Grid Engine (N1GE)
6 and its open source version as the job launchers for ORTE.

This commit was SVN r11153.
2006-08-10 21:46:52 +00:00
Ralph Castain
8496b6aff4 When a "fork" launch cannot find the executable, the system used to just return an error. This meant that the state of that process was never updated in the registry, leaving the counters at the incorrect levels. As a result, the triggers would never fire to indicate that the job had been aborted. This left orterun and other orteds/processes hanging.
This fix should fix the problem. I will test it on a broader range of systems forsooth...

This commit was SVN r11140.
2006-08-09 15:29:08 +00:00
Brian Barrett
16186978bb - Fix some compile issues in r11109
- indent / whitespace cleanup
- don't set --daemon-debug when pls debug is given, as it seems to make
  the daemons abort.

This commit was SVN r11113.

The following SVN revision numbers were found above:
  r11109 --> open-mpi/ompi@da7df6d257
2006-08-03 18:51:42 +00:00
Galen Shipman
da7df6d257 monitor bproc node state and terminate the job if a node in our job goes
down.. 

This commit was SVN r11109.
2006-08-03 05:29:49 +00:00
Ralph Castain
8bec270f90 Fix a bug noted by Jeff - we were no longer accurately recording in the registry that a process had been terminated when the user initiated the "kill" process (via cntrl-c).
Added another system-level test function for ORTE that just spins until terminated by a ctrl-c signal.

Modified orterun - added a couple of newlines to the output when abnormally terminating so the prompt always is on a new line.

This commit was SVN r10866.
2006-07-18 14:42:27 +00:00
Jeff Squyres
ffddfc5629 Turns out that it's a really Bad Idea(tm) to tm_spawn() and then not
keep the resulting tm_event_t that is generated because the back-end
TM library actually caches a bunch of stuff on it for internal
processing, and doesn't let go of it until tm_poll().

tm_event_t's are similar to (but slightly different than)
MPI_Requests: you can't do a million MPI_Isend()'s on a single
MPI_Request -- a) you need an array of MPI_Request's to fill and b)
you need to keep them around until all the requests have completed.

This commit was SVN r10820.
2006-07-14 22:04:41 +00:00
Jeff Squyres
ef8433a60b After more discussion on the phone, it seems easier to not muck around
in special components but rather go down to a /tmp branch.  So
removing these components and I'll branch next.

This commit was SVN r10771.
2006-07-12 22:12:29 +00:00
Jeff Squyres
62c189ea1c Fix a few blanket search/replaces
This commit was SVN r10768.
2006-07-12 21:54:05 +00:00
Jeff Squyres
36ca7497d1 Update m4 and configure files
This commit was SVN r10764.
2006-07-12 20:55:39 +00:00
Ralph Castain
a84898316c Create new components to support Thunderbird scalability development
This commit was SVN r10762.
2006-07-12 20:28:23 +00:00
Brian Barrett
41e144c879 Fix for ticket #92, bproc stdin being borked. The problem was that we were
using a pty for everything, which drops all buffered data on the floor when
close() is called on the daemon side, meaning EOF has some issues.  Instead,
do the same thing we do for other starters that use the fork() pls -- use
a pipe/fifo for stdin and stderr and a pty for stdout.  This is good enough
for what we need and avoids most of the issues with ptys.

This commit was SVN r10692.
2006-07-08 21:18:24 +00:00
Jeff Squyres
3d5d0959fa Remove unused variable, and therefore silence a compiler warning.
This commit was SVN r10673.
2006-07-06 10:44:04 +00:00
Josh Hursey
b1da6f8bc4 A bit more cleanup for that last patch.
* num_children should really be an int instead of size_t
  since 'size_t' is not signed and num_children can (in rare cases)
  drop below 0, and don't want it to roll around to MAX_INT or some
  such.

 * I figured out that this problem only happened to me because I use
  the pls_fork_reap_timeout MCA parameter and thus the only time that
  the code in pls_fork_module.c to waitpid is executed is if this is
  not set to 0 (I had it set to 1 to give my procs time to exit). I
  adjusted the loop from while{...} to do{...}while; so that it is
  executed at least once for consistency.

 * de-register the SIGCHILD callback for the pid before we attempt
  to kill it, so that we don't leave the door open for both the
  waitpids (the one in the callback, and the one in this function)
  to race to see who can wait on the child.

 * Move the 'thread release' to outside the for loop for a bit of an
  optimization, and always set the value to 0 since we want to 
  finish after this function.

 * Added a help message for the case when we can't send a kill()
   signal to the process. Should never happen, but all is possible
   in the wild wild west of HPC.

This commit was SVN r10666.
2006-07-05 21:38:23 +00:00
Josh Hursey
696bb4a0c0 A partial fix for the hanging orted bugs (Ticket #177)
When we force an application to terminate (via CTRL-C to mpirun)
we send an out-of-band message to the orted to reap its children.
the fork PLS was doing an internal waitpid but never releasing or
updating the information and signaling the condition variable. So
the fork PLS callback for SIGCHLD registered with the event library
and this waitpid are in a bit of a race to 'waitpid' for the children.
Since the PLS callback was the only one that handled the signal properly
when it 'won' then things were great -- as in the normal termination case.
But when it 'lost' -- as in the abnormal termination case -- the orted
never received the proper signal that its children had gone away.

We want to preserve the internal fork PLS callback since it allows
for a timeout while waiting for the child, which the event library
won't do.

This allows both to exist, and behave properly.

This was introduced in r9068.

The ticket is still open since the orted's hang in other situations
still. This is a fix for one of the causes.

This commit was SVN r10662.

The following SVN revision numbers were found above:
  r9068 --> open-mpi/ompi@c2c2daa966
2006-07-05 19:37:29 +00:00
Jeff Squyres
538965aeb0 Final merge of stuff from /tmp/tm-stuff tree (merged through
/tmp/tm-merge).  Validated by RHC.  Summary:

- Add --nolocal (and -nolocal) options to orterun
- Make some scalability improvements to the tm pls

This commit was SVN r10651.
2006-07-04 20:12:35 +00:00
Josh Hursey
5c5ce7e051 When 'mca_oob_send_callback' accesses the callback 'orte_pls_rsh_terminate_job_cb'
with an error status (< 0) then the req buffer is NULL. Put checks around the
OBJ_RELEASE(req) calls so that we don't try to release NULL :/

This commit was SVN r10641.
2006-07-03 22:44:54 +00:00
Josh Hursey
d082a63734 Add some new OPAL functionality.
After seeing the uglyness that is removing directories in the
codebase I decided to push down this to the OPAL by extending the
opal/os_create_dirpath.(c|h) to contain some more functionality.

In this process I renamed 'os_create_dirpath' to 'os_dirpath' since it
is a bit more general now.

Added a few functions to:
 - check if an directory is empty
 - check to see if the access permissions are set correctly
 - destroy the directory at the end of the dirpath
   - By using a caller callback function (a la Perl, I believe)
     for every file, the caller can have fine grained control over
     whether a specific file is deleted or not.

This simplifies things a bit for orte_session_dir_(finalize|cleanup)
as it should no longer contain any of this functionality, but uses
these functions to do the work.

From the external perspective nothing has changed, from the 
developer point of view we have some cleaner, more generic code.

This commit was SVN r10640.
2006-07-03 22:23:07 +00:00
Brian Barrett
0bd5acc51f * Fix for bus error in XGrid starter
This commit was SVN r10615.
2006-07-01 16:16:46 +00:00
Brian Barrett
2cf73912e2 * fix for signal forwarding additions in bproc_orted code
This commit was SVN r10529.
2006-06-27 19:59:07 +00:00
Sushant Sharma
76926756d0 variable ntid not being assigned any value was resulting in errors
This commit was SVN r10480.
2006-06-22 18:00:54 +00:00
Sushant Sharma
b5a16b6515 Updated xcpu launcher. open-mpi no longer needs xcpu library. Launcher code is now moved within xcpu.
This commit was SVN r10342.
2006-06-13 23:21:56 +00:00
Brian Barrett
17a8ccef89 * update XGrid API to match recent signal changes
This commit was SVN r10262.
2006-06-08 21:15:35 +00:00
Ralph Castain
ee5a626d25 Add ability to trap and propagate SIGUSR1/2 to remote processes. There are a number of small changes that hit a bunch of files:
1. Changed the RMGR and PLS APIs to add "signal_job" and "signal_proc" entry points. Only the "signal_job" entries are implemented - none of the components have implementations for "signal_proc" at this time. Thus, you can signal all of the procs in a job, but cannot currently signal only one specific proc.

2. Implemented those new API functions in all components except xgrid (Brian will do so very soon). Only the rsh/ssh and fork modules have been tested, however, and only under OS-X.

3. Added signal traps and callback functions for SIGUSR1/2 to orterun/mpirun that catch those signals and call the appropriate commands to propagate them out to all processes in the job.

4. Added a new test directory under the orte branch to (eventually) hold unit and system level tests for just the run-time. Since our test branch of the repository is under restricted access, people working on the RTE were continually developing their own system-level tests - thus making it hard to help diagnose problems. I have moved the more commonly-used functions here, and added one specifically for testing the SIGUSR1/2 functionality.

I will be contacting people directly to seek help with testing the changes on more environments. Other than compile issues, you should see absolutely no change in behavior on any of your systems - this additional functionality is transparent to anyone who does not issue a SIGUSR1/2 to mpirun.

Ralph

This commit was SVN r10258.
2006-06-08 18:27:17 +00:00
Jeff Squyres
4882dc0e2c Addendum to r9930: missed a chunk of the rsh pls to use the basename
of $libdir and $bindir (i.e., was correctly doing local launches, but
was still using $prefix/lib and $prefix/bin for remote launches).

[Re-]Fixes OFED bug 59.

This commit was SVN r10207.

The following SVN revision numbers were found above:
  r9930 --> open-mpi/ompi@1d6902296c
2006-06-05 21:12:36 +00:00
Jeff Squyres
1d6902296c Additions to the tm, slurm, and rsh pls modules to handle the --prefix
option as discussed on the devel-core mailing list.  The Big
Difference is that instead of hard-coding the strings "/lib" and
"/bin" in to append to the prefix, we append the basename of the local
libdir and bindir.  Hence, if your libdir is $prefix/lib64, we'll
append /lib64 to construct the remote node's LD_LIBRARY_PATH (etc.).

Also appended the orterun.1 man page to include a description of
--prefix, how it is constructed, what it handles / what it does not,
etc.

This commit was SVN r9930.
2006-05-16 14:14:12 +00:00
Gleb Natapov
80dfe7e39b remove newline from environment
This commit was SVN r9892.
2006-05-11 13:15:48 +00:00
Sushant Sharma
7a6e0c9ebf Fixed remote environment setup. Submitted by: Tim Woodall
This commit was SVN r9759.
2006-04-27 20:07:56 +00:00
Ralph Castain
95c4795157 Try a different tack...
This commit was SVN r9658.
2006-04-19 15:33:34 +00:00
Ralph Castain
93115fdaea Try again with passing the right enviro variables.
This commit was SVN r9629.
2006-04-13 18:07:22 +00:00
Ralph Castain
480af1c150 Add the missing enviro variables
This commit was SVN r9627.
2006-04-13 16:41:47 +00:00
Sushant Sharma
642e33fb3e xcpu launcher updated to setup the environment on remote nodes before launching jobs.
This commit was SVN r9622.
2006-04-12 22:42:41 +00:00
Ralph Castain
424900068f Update the xcpu launcher to setup the environment
This commit was SVN r9620.
2006-04-12 15:41:54 +00:00
Sushant Sharma
9fe5870862 xcpu pls component fixed so that it will compile correctly.
This commit was SVN r9617.
2006-04-11 20:27:13 +00:00
Ralph Castain
9adc16130e Proposed revision of the xcpu launcher to correctly incorporate the OpenRTE and Open MPI environment
This commit was SVN r9612.
2006-04-11 14:33:17 +00:00
Sushant Sharma
8d5289b2b8 Corrected Makefile.am files for pls and soh xcpu-components as per Brian's suggestion.
This commit was SVN r9519.
2006-04-03 17:14:47 +00:00
Jeff Squyres
858612fd06 Face the possibilty that the child may have already died.
This commit was SVN r9508.
2006-04-01 02:23:10 +00:00
Sushant Sharma
46f84b1e8e Added xcpu component in pls and soh.
This commit was SVN r9491.
2006-03-31 02:19:52 +00:00
Brian Barrett
23f0aef07c * fix casting warning
This commit was SVN r9407.
2006-03-24 02:52:34 +00:00
Tim Woodall
564c177922 corrections for threading support
This commit was SVN r9292.
2006-03-16 00:06:48 +00:00
Tim Woodall
648ca32742 correct include
This commit was SVN r9276.
2006-03-14 14:40:52 +00:00
Brian Barrett
c42da09796 * Fix a small bug George noticed - if you change the prefix (or any of the
installation directories) in configure, the files that depend on this
  information are not properly rebuilt.  If you need this information,
  don't setup a -D in the Makefile.am - instead, include 
  opal/install_dirs.h.
* Use the : option in AC_CONFIG_FILES to avoid needing to expose that
  we are playing around with temporary files with our headers to avoid
  rebuilding
* Clean up the version file information a bit, and like the install 
  directory stuff, make sure that there is a dependency so that 
  ompi_info gets rebuilt properly when a version number changes.

This commit was SVN r9256.
2006-03-12 04:35:01 +00:00
Brian Barrett
d0f5f8a242 * enable pty support for platforms that do not have openpty(), but do
have pty support in general.

This commit was SVN r9250.
2006-03-11 02:35:40 +00:00
Jeff Squyres
28a1610453 - Add mssing <netdb.h>
- Change #if to #ifdef

This commit was SVN r9146.
2006-02-26 16:06:58 +00:00
Jeff Squyres
b4765b6db6 Add <netdb.h>
This commit was SVN r9142.
2006-02-25 19:04:26 +00:00
Jeff Squyres
22da6ef4e4 This bit accidentally got lost in the testing of the bproc/fork path
and cwd update functionality.  For bproc, we *do* need to change
directories while checking the cwd because argv[0] may be expressed as
a relative path, and therefore needs to be checked from the cwd
expressed in the app context.

This commit was SVN r9084.
2006-02-17 16:15:21 +00:00
Jeff Squyres
2c91ac861a When not in debug mode, tie stdout/stderr to dev null so we don't see
messages from orted (i.e., from the srun command).

This commit was SVN r9083.
2006-02-17 15:06:08 +00:00
Jeff Squyres
81e0bd444b - Remove extraneous chdir("/tmp")
- For a local orted launch, chdir($HOME) to be consistent with what
  [we assume] will happen on the remote nodes

This commit was SVN r9073.
2006-02-16 22:14:05 +00:00
George Bosilca
ab59741df6 Access to environ is not granted for free, we have to declare it on MAC OS X
(at least with gcc 4.2).

This commit was SVN r9071.
2006-02-16 21:26:25 +00:00
Jeff Squyres
c2c2daa966 Change the behavior of orterun (mpirun, mpirexec) to search for
argv[0] and the cwd on the target node (i.e., the node where the
executable will be running in all systems except BProc, where the
searches are run on the node where orterun is invoked).
- fork pls now does cwd and argv[0] search in orted
- bproc pls does cwd and argv[0] search in orterun
- cwd behavior slightly different:
  - if user specifies a -wdir to orterun, we chdir() to there; if we
    can't for some reason, abort
  - if user does not specify a -wdir, try to chdir() to the dir where
    orterun was invoked.  If we can't for some reason (e.g., it
    doesn't exist on the target node), then try to chdir($HOME).  If
    we can't do that, then just live with whatever default directory
    we were put in.

This commit was SVN r9068.
2006-02-16 20:40:23 +00:00
Tim Woodall
039fe0ad29 change process group only in bproc case, as this is really
a workaround for a bproc4 bug

This commit was SVN r9064.
2006-02-16 16:19:37 +00:00