A subset of this patch needs to be applied to v1.2
Refs trac:928
This commit was SVN r15918.
The following Trac tickets were found above:
Ticket 928 --> https://svn.open-mpi.org/trac/ompi/ticket/928
Tie stdin to /dev/null to prevent stdin from being closed and thus making stdin not work in slurm allocations.
This commit was SVN r15892.
The following Trac tickets were found above:
Ticket 1047 --> https://svn.open-mpi.org/trac/ompi/ticket/1047
needs to override the default umask. By default, this is not used
since most environments do what the user would expect without any
help.
* Have TM use the newly added umask hook, so that processes inherit
the user's umask from mpirun rather than the pbs_mom's umask, which
the user has no control over.
This commit was SVN r15858.
Application Level Placement Scheduler (ALPS).
This commit was tested under two Cray machines at ORNL: Jaguar (Catamount)
and Rizzo (CNL Test cage). Both machines performed as they should across
the commit.
It is likely that mor changes will follow this the work and environment
stabilizes.
Most of the infrastructure works the same for Catamount and CNL
except for a few bits. Below are the highlights:
Default IFACE Change:
On Catamount we can use PTL_IFACE_DEFAULT, but on the CNL system we have access
to will fail on this interface, and should be set to:
IFACE_FROM_BRIDGE_AND_NALID(PTL_BRIDGE_UK,PTL_IFACE_SS).
So if we detect that we are running with YOD then use the former interface
and if we detect that we are running with ALPS then use the latter.
We will want to pursue a more elegant solution if this interface continues to
change across machines.
PtlGetId and cnos_register_ptlid:
The header suggests that these should never be called when launching with YOD.
But in the ALPS environment the cnos_barrier() will hang forever if these
functions are not called after PtlNIInit(). Since these functions only need to
be called once, and the orte rmgr/cnos component is loaded before the ompi
common/portals componet then just call these functions once in the rmgr/cnos
component.
cnos_barrier_init():
This is a noop for YOD, but critical for ALPS. So be sure to call it before
calling the first barrier in the rmgr/cnos component.
cnos_barrier vs cnos_pm_barrier:
It is suggested the cnos_pm_barrier only be used during finalization
as it will indicate to the launcher (yod or aprun) that the app is about
to complete. It was suggested that we use the regular cnos_barrier() instead.
I want to look into this a bit more to make sure there are not adverse
side effects. A note has been placed in the code to indicate this reasoning.
This commit was SVN r15756.
from orte_ns.compare_fields(), not 0 (yes, they're the same [today],
but it is much better to check for symbolic names...).
This commit was SVN r15731.
to light: we weren't ack'ing properly for streams that originated (or
originated via proxy) and terminated within the HNP. This commit
fixes that.
It also fixes a few style issues, and added some more opal_outputs for
debugging. Also, fixed a bug where the fact that we forwarded (and
therefore might need to update the ack) was not correctly reported if
there were multiple forwards (which there are not as the system is
currently using IOF, but there could be).
Refs trac:1098 -- want to get another pair of eyes to look at this before
I close the ticket.
This commit was SVN r15730.
The following Trac tickets were found above:
Ticket 1098 --> https://svn.open-mpi.org/trac/ompi/ticket/1098
int to void. This function call exit at the end, so there is no way to
return from there. Apply the same thing to the errmsg_abort function and
update all components.
This commit was SVN r15704.
- If one wants to use this solution, remember to unload the project 'orte-restart' which is currently not working for Windows.
This commit was SVN r15680.
grpcomm cnos component
- Remove the .ompi_ignore
- add a configure.m4 that should keep it from building on any system
other than Cray XT* (copied from rml/cnos)
- Fix some mis-named symbols resulting from cut/paste errors.
This patch brings the Cray build back into 'working' order.
This commit was SVN r15651.
or two.
* checking lsb_init() is not sufficient to know whether you're in an
LSF job or not; you also need to check for environment variable
markers
* remove lots of debugging output
* no need for the sds lsf to call lsb_init()
* remove some slurm-like dead code and a copy-n-paste error in the
sds lsf
This commit was SVN r15644.
is no need for the IP address in most cases (filem being one dubious
exception), so just publish and hand around the supposedly opaque contact
info strings
This commit was SVN r15638.
Remove unnecessary locks because functions that are calling this
function proper lock/unlock the orted_comm_mutex. Therefore this
unlocks cause some imballance.
This commit was SVN r15630.
in all cases. This is now done in the oob tcp open function.
As a result, the unregistering have to be done in the close
function.
This commit was SVN r15603.
r15390 - Changed the paradigm in which the runtime worked by enabling the mpirun
process to become an orted and spawn processes. This broke the C/R for this
special case as it required that the orted start the process, and that
the hierarchy remains.
The fix was to allow the global coordinator to be a local coordinator as well
for this case.
r15528 - Changed the selection logic for the RML. This caused the application to
segv if the 'ftrm' wrapper component was selected as it tried to modify a NULL
pointer.
The fix was to move the 'module swap' code into the init() function, and swap
when passed a NULL pointer. It sounds bad, but actually cleans up the code a bit
more.
Still have to fix the 'routed' framework.
This commit was SVN r15566.
The following SVN revision numbers were found above:
r15390 --> open-mpi/ompi@bd65f8ba88
r15528 --> open-mpi/ompi@39a6057fc6
* General TCP cleanup for OPAL / ORTE
* Simplifying the OOB by moving much of the logic into the RML
* Allowing the OOB RML component to do routing of messages
* Adding a component framework for handling routing tables
* Moving the xcast functionality from the OOB base to its own framework
Includes merge from tmp/bwb-oob-rml-merge revisions:
r15506, r15507, r15508, r15510, r15511, r15512, r15513
This commit was SVN r15528.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r15506
r15507
r15508
r15510
r15511
r15512
r15513
Cleanup ALL instances of output involving the printing of orte_process_name_t structures using the ORTE_NAME_ARGS macro so that the number of fields and type of data match. Replace those values with a new macro/function pair ORTE_NAME_PRINT that outputs a string (using the new thread safe data capability) so that any future changes to the printing of those structures can be accomplished with a change to a single point.
Note that I could not possibly find outputs that directly print the orte_process_name_t fields, but only dealt with those that used ORTE_NAME_ARGS. Hence, you may still have a few outputs that bark during compilation. Also, I could only verify those that fall within environments I can compile on, so other environments may yield some minor warnings.
This commit was SVN r15517.
build it's possible that we have to process an ack before this function
returns. If we don't release the lock here we cause a deadlock later
in ack processing function.
This commit was SVN r15441.
Fix the blasted iof null component so it only is selected if/when directed.
This commit was SVN r15437.
The following SVN revision numbers were found above:
r15390 --> open-mpi/ompi@bd65f8ba88
You will not see any impact from this change unless you use the syntax described in ticket #1023. I've tried as many of the RAS components as possible and saw no problem - there may be issues with other RAS components that would not compile on any of my systems. Anything that appears should be trivial to fix.
This commit was SVN r15427.
This fixes C/R support for the trunk which regressed in r15390 due to the RTE
changes.
This commit was SVN r15409.
The following SVN revision numbers were found above:
r15219 --> open-mpi/ompi@f88aa6c273
r15390 --> open-mpi/ompi@bd65f8ba88
We no longer store whether we are a singleton in a MCA parameter, we now use a global constant. So all references to the MCA parameter must be removed.
This commit was SVN r15408.
The following SVN revision numbers were found above:
r15390 --> open-mpi/ompi@bd65f8ba88
The problem stemmed from no longer launching a local orted on the same node as mpirun. The orted would save and reuse the base environment. Mpirun didn't do that, and the odls was using the orted's globally saved environment (which wasn't being set).
This fix establishes a globally accessible base launch environment that both the orted and mpirun can utilize. Since we now use that, we don't need to pass it to the odls_launch_proc function, so remove that param from the API (and modify all components to handle the change).
This commit was SVN r15405.
Short description: major changes include -
1. singletons now fork/exec a local daemon to manage their operations.
2. the orte daemon code now resides in libopen-rte
3. daemons no longer use the orte triggering system during startup. Instead, they directly call back to their parent pls component to report ready to operate. A base function to count the callbacks has been provided.
I have modified all the pls components except xcpu and poe (don't understand either well enough to do it). Full functionality has been verified for rsh, SLURM, and TM systems. Compile has been verified for xgrid and gridengine.
This commit was SVN r15390.
* Making some symbols and types be global (vs. static) in orterun
* Adding a "ddt" entry in the MCA parameter orte_base_user_debugger
default value
* Add support for @executable@, @executable_argv@, and @single_app@
tokens in the orte_base_user_debugger MCA parameter.
* Added various error checks and corresponding help messages after
finding a debugger in the PATH
Fixes trac:1081
This commit was SVN r15323.
The following Trac tickets were found above:
Ticket 1081 --> https://svn.open-mpi.org/trac/ompi/ticket/1081
VxWorks. Still some issues remaining, I'm sure.
Refs trac:1010
This commit was SVN r15320.
The following Trac tickets were found above:
Ticket 1010 --> https://svn.open-mpi.org/trac/ompi/ticket/1010
line may point to a few characters beyond the beginning of the buffer
(if the buffer had some extra white space padding at the beginning).
So if we want to free the buffer, free full_line, not line.
This commit was SVN r15315.
types and add STOP_AT_FIRST_PRIORITY type for framework configuration,
which allows all components at the highest priority that succeeds to
succeed
* Use STOP_AT_FIRST_PRIORITY type for gpr framework, so that the null
component isn't built when the replica and proxy components are
available.
This commit was SVN r15286.
* Make orted.1 man page be non-descriptive because it's really an
internal command.
* Re-work the opal_wrapper man page logic a bit so that we can have a
real opal_wrapper.1 installed that says "don't look here -- look at
mpicc (etc.)"
This commit was SVN r15264.
The default odls has been updated and works fine. The process odls has been updated, but I could not verify its operation. The bproc ODLS has not been updated yet. Ralph will look at it soon.
This commit was SVN r15257.
* Remove the 'opal_mca_base_param_use_amca_sets' global variable
* Harness the fact that you can (read should) call the cmd_line functions
before initializing opal_init_util(). This pushes the MCA/GMCA/AMCA
command line options into the environment before OPAL inits and starts
to use these values. By putting the cmd_line parse before opal_init_util
in orterun and orted we only parse the *MCA parameter files once, and
correctly (alleviating the need to 'recache' the files on init.)
* Small bits of cleanup.
This commit was SVN r15219.
Per suggestion, if we don't find a valid shell via getpwuid(), also
check the $SHELL environment variable. Also perform a few minor
cleanups along the way.
This commit was SVN r15156.
The following Trac tickets were found above:
Ticket 1060 --> https://svn.open-mpi.org/trac/ompi/ticket/1060
(and, due to lack of in code documentation, in r14661).
The {{{opal_mca_base_param_use_amca_sets}}} flag tells the orted that it should
not look at the parameter files just yet since it may have an AMCA parameter
file to look at first. So we need to set this to {{{false}}} before initializing
the MCA paras, then quickly turn around and re-init them when we have the full
information.
This commit fixes trac:1058
This commit was SVN r15144.
The following SVN revision numbers were found above:
r14449 --> open-mpi/ompi@0ba47105ed
r14661 --> open-mpi/ompi@df86202202
The following Trac tickets were found above:
Ticket 1058 --> https://svn.open-mpi.org/trac/ompi/ticket/1058
The SnapC Full local Coordinator used this argument to attach to the job the
daemon would be launching. So once this option was removed C/R support broke.
This commit has the local coordinator attach to the job just before it is
launched by the ODLS module. This is a much cleaner solution, and will
eventually allow the SnapC modules to attach to multiple jobs launched
on a single machine.
This commit fixes the C/R regression introduced in r15007.
This commit was SVN r15121.
The following SVN revision numbers were found above:
r15007 --> open-mpi/ompi@85df3bd92f
the multiple threads accessing the OOB/registry asynchronously via the
callbacks. The quickest solution (but definitively not the cleanest) is
to serialize these callbacks in such a way that at any given time
only one thread can execute a callbacks.
This commit was SVN r15086.
through the win dll using multiple threads, we have to insure that
the oob callbacks happens only in a synchronous way or really bad
things happens with the current design (blocking messages from a receive
callback).
This commit was SVN r15069.
The warning was indicative of overly-complex code anyway. So I
removed the "first" bool and simply use a sentinel value in seq_min to
indicate that nothing has changed. Note that this is "correct enough"
for the moment -- more fixes will come in this area with tickets #1049
and/or #1051.
This commit was SVN r15013.
single threaded builds. In its default configuration, all this does
is ensure that there's at least a good chance of threads building
based on non-threaded development (since the variable names will be
checked). There is also code to make sure that a "mutex" is never
"double locked" when using the conditional macro mutex operations.
This is off by default because there are a number of places in both
ORTE and OMPI where this alarm spews mega bytes of errors on a
simple test. So we have some work to do on our path towards
thread support.
Also removed the macro versions of the non-conditional thread locks,
as the only places they were used, the author of the code intended
to use the conditional thread locks. So now you have upper-case
macros for conditional thread locks and lowercase functions for
non-conditional locks. Simple, right? :).
This commit was SVN r15011.
1. generalize orte_rml.xcast to become a general broadcast-like messaging system. Messages can now be sent to any tag on the daemons or processes. Note that any message sent via xcast will be delivered to ALL processes in the specified job - you don't get to pick and choose. At a later date, we will introduce an augmented capability that will use the daemons as relays, but will allow you to send to a specified array of process names.
2. extended orte_rml.xcast so it supports more scalable message routing methodologies. At the moment, we support three: (a) direct, which sends the message directly to all recipients; (b) linear, which sends the message to the local daemon on each node, which then relays it to its own local procs; and (b) binomial, which sends the message via a binomial algo across all the daemons, each of which then relays to its own local procs. The crossover points between the algos are adjustable via MCA param, or you can simply demand that a specific algo be used.
3. orteds no longer exhibit two types of behavior: bootproxy or VM. Orteds now always behave like they are part of a virtual machine - they simply launch a job if mpirun tells them to do so. This is another step towards creating an "orteboot" functionality, but also provided a clean system for supporting message relaying.
Note one major impact of this commit: multiple daemons on a node cannot be supported any longer! Only a single daemon/node is now allowed.
This commit is known to break support for the following environments: POE, Xgrid, Xcpu, Windows. It has been tested on rsh, SLURM, and Bproc. Modifications for TM support have been made but could not be verified due to machine problems at LANL. Modifications for SGE have been made but could not be verified. The developers for the non-verified environments will be separately notified along with suggestions on how to fix the problems.
This commit was SVN r15007.
structures in the system. Instead of using memcmp, use the ns function.
This won't cause a problem as long as all three elements of the name are
ints, but if they have different sizes, alignment and padding rules
can cause memcmp() to compare padding space, which rarely holds a sane
value.
This commit was SVN r14998.
framework. Updated pointers to match current definitions.
* Trimmed some dead wood while I was at it:
* No need for component close function that does nothing
* Use BEGIN/END_C_DECLS
* Use recent MCA param register function
* Ditch MCA param orte_iof_debug (it wasn't used anywhere)
* Use MCA param orte_iof_override properly in the code (i.e., look
up the value once and use the cached value later)
This commit was SVN r14981.
