1
1

465 Коммитов

Автор SHA1 Сообщение Дата
Ralph Castain
5a24d6f60e Remove an option that the orteds don't actually support...
This commit was SVN r22027.
2009-09-29 02:08:27 +00:00
Ralph Castain
c749fefbd0 Instead of an odls-base mca param, make report_bindings a global param so that we can (a) detect it was set in the plm, and then (b) ensure it gets passed along to remote orteds so they will comply with the request.
This commit was SVN r22021.
2009-09-28 03:17:15 +00:00
Ralph Castain
ef0fd8b8d1 Return an error code if the job failed to start
This commit was SVN r22019.
2009-09-26 03:34:58 +00:00
Ralph Castain
5fb3d13c24 Cleanup some pointer array addressing
This commit was SVN r21963.
2009-09-09 20:59:17 +00:00
Ralph Castain
51b13b3d5c A few minor cleanups in where threads are unlocked.
Reset mpirun's exit code when we restart failed procs

This commit was SVN r21955.
2009-09-09 05:31:06 +00:00
Shiqing Fan
fb777134cf Adjust the command string length.
This commit was SVN r21905.
2009-08-27 13:42:55 +00:00
Shiqing Fan
ffd55631bc Deal with the case when the prefix is NULL.
This commit was SVN r21902.
2009-08-27 13:11:18 +00:00
Shiqing Fan
6e57772ebd Remove a few redundant security functions, and use USERDOMAIN environment variable for the domain name.
This commit was SVN r21898.
2009-08-27 08:00:49 +00:00
Rainer Keller
8e1b23779f - Replace combinations of
#if defined (c_plusplus)
          defined (__cplusplus)
   followed by
      extern "C" {
   and the closing counterpart by BEGIN_C_DECLS and END_C_DECLS.

   Notable exceptions are:
    - opal/include/opal_config_bottom.h:
      This is our generated code, that itself defines BEGIN_C_DECL and
      END_C_DECL
    - ompi/mpi/cxx/mpicxx.h:
      Here we do not include opal_config_bottom.h:                                 
    - Belongs to external code:                                                    
      opal/mca/backtrace/darwin/MoreBacktrace/MoreDebugging/MoreBacktrace.c        
      opal/mca/backtrace/darwin/MoreBacktrace/MoreDebugging/MoreBacktrace.h        
    - opal/include/opal/prefetch.h:
      Has C++ specific macros that are protected:                                  

    - Had #if ... } #endif  _and_ END_C_DECLS (aka end up with 2x
      END_C_DECLS)
      ompi/mca/btl/openib/btl_openib.h
    - opal/event/event.h has #ifdef __cplusplus as BEGIN_C_DECLS...
    - opal/win32/ompi_process.h: had extern "C"\n {...
      opal/win32/ompi_process.h: dito
    - ompi/mca/btl/pcie/btl_pcie_lex.l: needed to add *_C_DECLS
      ompi/mpi/f90/test/align_c.c: dito
    - ompi/debuggers/msgq_interface.h: used #ifdef __cplusplus
    - ompi/mpi/f90/xml/common-C.xsl: Amend

   Tested on linux using --with-openib and --with-mx

   The following do not contain either opal_config.h, orte_config.h or
   ompi_config.h
   (but possibly other header files, that include one of the above):
      ompi/mca/bml/r2/bml_r2_ft.h
      ompi/mca/btl/gm/btl_gm_endpoint.h
      ompi/mca/btl/gm/btl_gm_proc.h
      ompi/mca/btl/mx/btl_mx_endpoint.h
      ompi/mca/btl/ofud/btl_ofud_endpoint.h
      ompi/mca/btl/ofud/btl_ofud_frag.h
      ompi/mca/btl/ofud/btl_ofud_proc.h
      ompi/mca/btl/openib/btl_openib_mca.h
      ompi/mca/btl/portals/btl_portals_endpoint.h
      ompi/mca/btl/portals/btl_portals_frag.h
      ompi/mca/btl/sctp/btl_sctp_endpoint.h
      ompi/mca/btl/sctp/btl_sctp_proc.h
      ompi/mca/btl/tcp/btl_tcp_endpoint.h
      ompi/mca/btl/tcp/btl_tcp_ft.h
      ompi/mca/btl/tcp/btl_tcp_proc.h
      ompi/mca/btl/template/btl_template_endpoint.h
      ompi/mca/btl/template/btl_template_proc.h
      ompi/mca/btl/udapl/btl_udapl_eager_rdma.h
      ompi/mca/btl/udapl/btl_udapl_endpoint.h
      ompi/mca/btl/udapl/btl_udapl_mca.h
      ompi/mca/btl/udapl/btl_udapl_proc.h
      ompi/mca/mtl/mx/mtl_mx_endpoint.h
      ompi/mca/mtl/mx/mtl_mx.h
      ompi/mca/mtl/psm/mtl_psm_endpoint.h
      ompi/mca/mtl/psm/mtl_psm.h
      ompi/mca/pml/cm/pml_cm_component.h
      ompi/mca/pml/csum/pml_csum_comm.h
      ompi/mca/pml/dr/pml_dr_comm.h
      ompi/mca/pml/dr/pml_dr_component.h
      ompi/mca/pml/dr/pml_dr_endpoint.h
      ompi/mca/pml/dr/pml_dr_recvfrag.h
      ompi/mca/pml/example/pml_example.h
      ompi/mca/pml/ob1/pml_ob1_comm.h
      ompi/mca/pml/ob1/pml_ob1_component.h
      ompi/mca/pml/ob1/pml_ob1_endpoint.h
      ompi/mca/pml/ob1/pml_ob1_rdmafrag.h
      ompi/mca/pml/ob1/pml_ob1_recvfrag.h
      ompi/mca/pml/v/pml_v_output.h
      opal/include/opal/prefetch.h
      opal/mca/timer/aix/timer_aix.h
      opal/util/qsort.h
      test/support/components.h

This commit was SVN r21855.

The following SVN revision numbers were found above:
  r2 --> open-mpi/ompi@58fdc18855
2009-08-20 11:42:18 +00:00
Ralph Castain
dbb3cbe3dd Fix a reported problem when specifying orte_launch_agent - if only one word was given, we inadvertently appended a "NULL" to the end of the cmd.
This commit was SVN r21827.
2009-08-18 14:57:34 +00:00
Ralph Castain
3f3b46495e Add some error checking to the tm launcher
This commit was SVN r21818.
2009-08-14 03:13:02 +00:00
Rainer Keller
cea3d68ef6 - Fix reference counting of daemons killed.
This commit was SVN r21810.
2009-08-12 14:04:50 +00:00
Shiqing Fan
bce2f44154 Update related .windows files with proper compiling properties, in order to have a successful DSO build.
This commit was SVN r21805.
2009-08-12 08:55:58 +00:00
Ralph Castain
1dc12046f1 Modify the OMPI paffinity and mapping system to support socket-level mapping and binding. Mostly refactors existing code, with modifications to the odls_default module to support the new capabilities.
Adds several new mpirun options:

* -bysocket - assign ranks on a node by socket. Effectively load balances the procs assigned to a node across the available sockets. Note that ranks can still be bound to a specific core within the socket, or to the entire socket - the mapping is independent of the binding.

* -bind-to-socket - bind each rank to all the cores on the socket to which they are assigned.

* -bind-to-core - currently the default behavior (maintained from prior default)

* -npersocket N - launch N procs for every socket on a node. Note that this implies we know how many sockets are on a node. Mpirun will determine its local values. These can be overridden by provided values, either via MCA param or in a hostfile

Similar features/options are provided at the board level for multi-board nodes.

Documentation to follow...

This commit was SVN r21791.
2009-08-11 02:51:27 +00:00
Rainer Keller
76469ea64a - Change the property of a few files, that obviously
don't need to be svn:executable...

This commit was SVN r21786.
2009-08-11 01:40:00 +00:00
Rainer Keller
784b9b9f5b - Based and updated from Ken's patch: since CLE-2.1 does not offer
the BATCH_PARTITION_ID anymore, use the ras-alps-command.sh script to
   figure out the jobs ID to query from ALPS.

   Gracefully report errors, update the help file and parse the sysconfig file

This commit was SVN r21772.
2009-08-07 01:15:09 +00:00
Shiqing Fan
3e24d3df70 An ORTE event fix for Windows, i.e. using socket pairs instead of pipes on Windows.
This commit was SVN r21726.
2009-07-22 07:39:52 +00:00
Ralph Castain
6c85d954f3 Use a conditioned wait to serialize launches when they come from multiple sources (e.g., an orte application that spawns multiple jobs).
This commit was SVN r21718.
2009-07-20 01:51:29 +00:00
Ralph Castain
1a5f7245c8 Create a new message handling method for serializing responses. Place recvd messages on a list, using a file descriptor and the event library to trigger processing. This is identical in design to what is used in the IOF.
Use it first in the plm_base_receive to serialize multiple comm_spawn and update_proc requests.

This commit was SVN r21717.
2009-07-19 18:07:04 +00:00
Ralph Castain
1d74ab6e3c Cleanup some pointer array addressing and ensure we always exit the function cleanly
This commit was SVN r21716.
2009-07-19 18:05:04 +00:00
Ralph Castain
c3ce908515 Shift a debug output to come at a better place
This commit was SVN r21715.
2009-07-19 17:56:48 +00:00
Ralph Castain
210f591f1c Cleanup array addressing for opal_pointer_array
This commit was SVN r21710.
2009-07-17 22:20:30 +00:00
Ralph Castain
51a8b89a83 Treat termination of continuously operating processes as an abort
This commit was SVN r21709.
2009-07-17 22:20:05 +00:00
Ralph Castain
4c1eb040b0 Enable the system to keep functioning even when multiple launches are occurring simultaneously.
This is a bit of a hack, but it does seem to allow the system to work. A better solution is being discussed.

This commit was SVN r21705.
2009-07-17 02:28:47 +00:00
George Bosilca
3e971e61f3 The system headers are supposed to be protected by #ifdef and not by #if.
This commit was SVN r21700.
2009-07-16 18:27:33 +00:00
Ralph Castain
dbac602be5 Add support for the add-host and add-hostfile MPI Info keys to allow Comm_spawn users to add new hosts to those already known by mpirun.
Requires full testing once comm_spawn is fixed (Edgar is working that now).

This commit was SVN r21664.
2009-07-14 14:34:11 +00:00
Ralph Castain
60edbc7220 Fix hetero operations and comm_spawn (to a point).
Remove all architecture references from ORTE and put them back in the modex using modex_send/recv calls.

Hetero operations are now fully supported again. Comm_spawn now works up to the point where it segfaults due to an error in the CID code - which now allows Edgar to dig further! :-)

This commit was SVN r21655.
2009-07-13 20:03:41 +00:00
Ralph Castain
1b418dd397 Fix segfault in comm_spawn. The underlying problem breaking comm_spawn, however, remains - the change to make modex non-blocking causes the system to fail due to the arch not getting properly set.
Fix for that coming shortly.

This commit was SVN r21646.
2009-07-13 15:13:06 +00:00
Ralph Castain
b97f885c00 Restore the original API to terminate individual processes instead of the entire job. This was originally removed as we didn't at that time know how to take advantage of it. Some of us are now working on proactive resilience methods that move procs prior to node failure, so this is now a required API. Modify the odls, plm, and orted functions to support this new functionality.
Continue work on the resilient mapper, completing support for fault groups.

This commit was SVN r21639.
2009-07-13 02:29:17 +00:00
Ralph Castain
9635db373d Ensure that we properly exit if the executable isn't found
This commit was SVN r21570.
2009-07-01 03:16:13 +00:00
Shiqing Fan
656ec00611 Remove the definition of _WIN32_DCOM. It only enables DCOM when _WIN32_WINNT is less then 0x400 (Win 95, 98), and we are supporting 0x502(Win XP) and above in Open MPI. Thanks George for pointing this out.
This commit was SVN r21555.
2009-06-27 23:36:25 +00:00
Ralph Castain
d0a5468deb Continue gradual cleanup of pointer array addressing.
This commit was SVN r21550.
2009-06-26 22:40:28 +00:00
Ralph Castain
b96a71b62e Enable restart of individual processes upon command via the errmgr callback function. It needs an external application to drive this capability, so normal operations shouldn't be affected.
Does not support MPI applications. More work coming to update daemon accounting on movement of procs across nodes.

This commit was SVN r21545.
2009-06-26 20:54:58 +00:00
Shiqing Fan
b0b11e8465 Clean up things properly.
This commit was SVN r21539.
2009-06-26 15:20:19 +00:00
Shiqing Fan
6ba25951b4 Add a separate function for reading remote registry, so that it could be easily reused.
Add two options for plm process module, i.e. remote_env_prefix for getting OMPI prefix on remote computer by reading its user environment variable (OPENMPI_HOME), and remote_reg_prefix is similar, but it reads the registry on the remote computer. Reading remote env prefix has a higher priority than reading reg prefix, so that user can use their own installation of OMPI.

This commit was SVN r21538.
2009-06-26 13:35:45 +00:00
Shiqing Fan
5b4b8f8899 Get rid of a bunch of compiler warnings on Windows.
This commit was SVN r21536.
2009-06-26 07:51:39 +00:00
Josh Hursey
26363fcc8b Move the end timer to the end of the launch loop for the HNP. This matches the timing in the SLURM component, and makes the timing work when enabling the 'tree' based rsh launch.
This commit was SVN r21528.
2009-06-25 13:28:34 +00:00
Josh Hursey
8705c7edb1 Fix {{{orte_timing}}} for the RSH component of the PLM.
It never set the daemon launch counter before launching.
In the output for 'total job launch time' make the message match that of the SLURM PLM for easier parsing.

This commit was SVN r21527.
2009-06-25 12:37:58 +00:00
Shiqing Fan
07f7fe1a1b Add two CCP params, for user specifying stdout and stderr to files.
This commit was SVN r21521.
2009-06-25 09:03:50 +00:00
George Bosilca
fabebd140f If the daemon doesn't contain orted we're adding the prefix_dir and
bin_base twice. Create a temporary full_orted_cmd to cope with this
case.

This commit was SVN r21519.
2009-06-24 23:57:32 +00:00
George Bosilca
32f632ad5a Put back the multi-word command line argument for the RSH PLM.
This commit was SVN r21518.
2009-06-24 23:51:53 +00:00
George Bosilca
c0750343b5 Partially revert 21513. Beware of the exception on the orte_launch_agent which
is treated apart in orte_plm_base_setup_orted_cmd.

This commit was SVN r21517.
2009-06-24 23:48:14 +00:00
Ralph Castain
2e98ba3fd0 Complete implementation of regexp launch with static oob ports. Only enabled for SLURM at this time - migration to Torque coming
This commit was SVN r21516.
2009-06-24 20:31:26 +00:00
George Bosilca
53e76eed75 Missed an include.
This commit was SVN r21515.
2009-06-24 20:19:47 +00:00
George Bosilca
35033aeae9 Allow the PLM RSH to follow the routing table with the spawn, instead of the hard
coded binomial approach. Now, the PLM extract the children information from the
routed component, and the startup follow the routed topology. As a side effect,
we can now launch using linear or radix topologies, in addition to the previously
binomial topology.

This commit was SVN r21514.
2009-06-24 19:56:29 +00:00
George Bosilca
8cb8f28d9d When we get a report from an orted about its state, don't use the sender of
the message to update the structures, but instead use the information from
the URI. The reason is that even the launch report messages can get routed.

Deal with the orted_cmd_line in a single location.

This commit was SVN r21513.
2009-06-24 19:51:52 +00:00
Shiqing Fan
6750657a97 Change the way that we detect cluster nodes status and clean up the CCP components.
Normally, any non-windows nodes should return  NodeStatus_Unreachable, but that's not true. We have noticed that Scientific Linux cluster nodes, which are in the same subnet as the Windows nodes, return NodeStatus_PendingApproval value, and the RAS CCP takes them as Windows nodes. So let's just use the "Ready" nodes.

This commit was SVN r21510.
2009-06-24 19:15:15 +00:00
Shiqing Fan
9bdf258d3c Add an initial support for remote process launch using WMI (Windows Management Instrumentation) in the environment where Windows 2003/2008 server and HPC pack are not installed. Users in a domain will be able to use domain computers as a cluster with their domain Credentials, the computers may run Windows XP or higher, e.g. Vista, and/or Windows 7 (later), with correct settings and permissions for WMI and Windows firewall.
This commit was SVN r21508.
2009-06-24 18:59:24 +00:00
George Bosilca
addaf7aaf8 Repair the tree spawn. The problem seems to come from the fact
that now the HNP send the messages using the routed component. In the case
of tree spawn, when a intermediary node spawn a child it doesn't know how
to forward a message to it, so when the node-map message is coming from
the HNP (as there is nothing yet in the contact/routing table) the message
is sent back the way it came. As a result the node-map message keeps jumping
between the HNP and the first level orteds.

The solution is to add a new option to the children orte_parent_uri, which
is only set when the orted is _not_ directly spawned by the HNP. When this
option is present on the argument list, the orted will add the parent to
its routing, and force the parent to update his routes (by sending the URI).
With this approach, the routing tree is build in same time as the processes
are spawned, and all messages from the HNP can be routed to the leaves.

However, this is far from an optimal solution. Right now, this so called tree
spawn, only spawn the children in a tree without doing anything about the
"connect back to the HNP" step. The HNP is flooded with reports from all the
orted. The total number of messages is higher than in the non tree startup
scheme, so we do not expect this approach to be scalable in the current
incarnation. A complete overhaul of the tree startup is required in order
improve the scalability. Stay tuned!

This commit was SVN r21504.
2009-06-23 22:10:25 +00:00
Ralph Castain
0ba845fed2 Continue development of regular expression support by implementing it for slurm launches. Works for both initial (cmd line and non-cmd line) and comm_spawn launch.
Additional work required to fully enable static port support when using cmd line regular expression launch system.

This commit was SVN r21502.
2009-06-23 20:25:38 +00:00