2008-02-28 04:57:57 +03:00
|
|
|
/*
|
2010-03-13 02:57:50 +03:00
|
|
|
* Copyright (c) 2004-2010 The Trustees of Indiana University and Indiana
|
2008-02-28 04:57:57 +03:00
|
|
|
* University Research and Technology
|
|
|
|
|
* Corporation. All rights reserved.
|
2008-02-28 08:32:23 +03:00
|
|
|
* Copyright (c) 2004-2008 The University of Tennessee and The University
|
2008-02-28 04:57:57 +03:00
|
|
|
* of Tennessee Research Foundation. All rights
|
|
|
|
|
* reserved.
|
|
|
|
|
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
|
|
|
|
* University of Stuttgart. All rights reserved.
|
|
|
|
|
* Copyright (c) 2004-2005 The Regents of the University of California.
|
|
|
|
|
* All rights reserved.
|
2009-02-13 07:14:10 +03:00
|
|
|
* Copyright (c) 2007-2009 Cisco Systems, Inc. All rights reserved.
|
2010-08-09 23:28:56 +04:00
|
|
|
* Copyright (c) 2009-2010 Oracle and/or its affiliates. All rights reserved.
|
2008-02-28 04:57:57 +03:00
|
|
|
* $COPYRIGHT$
|
|
|
|
|
*
|
|
|
|
|
* Additional copyrights may follow
|
|
|
|
|
*
|
|
|
|
|
* $HEADER$
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include "orte_config.h"
|
|
|
|
|
#include "orte/constants.h"
|
|
|
|
|
#include "orte/types.h"
|
|
|
|
|
|
|
|
|
|
#ifdef HAVE_SYS_TIME_H
|
|
|
|
|
#include <sys/time.h>
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
#include "opal/mca/base/mca_base_param.h"
|
2009-08-11 06:51:27 +04:00
|
|
|
#include "opal/mca/paffinity/paffinity.h"
|
2008-09-01 21:49:31 +04:00
|
|
|
#include "opal/util/argv.h"
|
2009-02-14 05:26:12 +03:00
|
|
|
#include "opal/util/output.h"
|
2010-05-18 03:08:56 +04:00
|
|
|
#include "opal/util/opal_sos.h"
|
2008-02-28 08:32:23 +03:00
|
|
|
#include "opal/class/opal_pointer_array.h"
|
2009-06-17 06:54:20 +04:00
|
|
|
#include "opal/class/opal_value_array.h"
|
2008-02-28 04:57:57 +03:00
|
|
|
#include "opal/dss/dss.h"
|
2010-04-23 08:44:41 +04:00
|
|
|
#include "opal/threads/threads.h"
|
This commit represents a bunch of work on a Mercurial side branch. As
such, the commit message back to the master SVN repository is fairly
long.
= ORTE Job-Level Output Messages =
Add two new interfaces that should be used for all new code throughout
the ORTE and OMPI layers (we already make the search-and-replace on
the existing ORTE / OMPI layers):
* orte_output(): (and corresponding friends ORTE_OUTPUT,
orte_output_verbose, etc.) This function sends the output directly
to the HNP for processing as part of a job-specific output
channel. It supports all the same outputs as opal_output()
(syslog, file, stdout, stderr), but for stdout/stderr, the output
is sent to the HNP for processing and output. More on this below.
* orte_show_help(): This function is a drop-in-replacement for
opal_show_help(), with two differences in functionality:
1. the rendered text help message output is sent to the HNP for
display (rather than outputting directly into the process' stderr
stream)
1. the HNP detects duplicate help messages and does not display them
(so that you don't see the same error message N times, once from
each of your N MPI processes); instead, it counts "new" instances
of the help message and displays a message every ~5 seconds when
there are new ones ("I got X new copies of the help message...")
opal_show_help and opal_output still exist, but they only output in
the current process. The intent for the new orte_* functions is that
they can apply job-level intelligence to the output. As such, we
recommend that all new ORTE and OMPI code use the new orte_*
functions, not thei opal_* functions.
=== New code ===
For ORTE and OMPI programmers, here's what you need to do differently
in new code:
* Do not include opal/util/show_help.h or opal/util/output.h.
Instead, include orte/util/output.h (this one header file has
declarations for both the orte_output() series of functions and
orte_show_help()).
* Effectively s/opal_output/orte_output/gi throughout your code.
Note that orte_output_open() takes a slightly different argument
list (as a way to pass data to the filtering stream -- see below),
so you if explicitly call opal_output_open(), you'll need to
slightly adapt to the new signature of orte_output_open().
* Literally s/opal_show_help/orte_show_help/. The function signature
is identical.
=== Notes ===
* orte_output'ing to stream 0 will do similar to what
opal_output'ing did, so leaving a hard-coded "0" as the first
argument is safe.
* For systems that do not use ORTE's RML or the HNP, the effect of
orte_output_* and orte_show_help will be identical to their opal
counterparts (the additional information passed to
orte_output_open() will be lost!). Indeed, the orte_* functions
simply become trivial wrappers to their opal_* counterparts. Note
that we have not tested this; the code is simple but it is quite
possible that we mucked something up.
= Filter Framework =
Messages sent view the new orte_* functions described above and
messages output via the IOF on the HNP will now optionally be passed
through a new "filter" framework before being output to
stdout/stderr. The "filter" OPAL MCA framework is intended to allow
preprocessing to messages before they are sent to their final
destinations. The first component that was written in the filter
framework was to create an XML stream, segregating all the messages
into different XML tags, etc. This will allow 3rd party tools to read
the stdout/stderr from the HNP and be able to know exactly what each
text message is (e.g., a help message, another OMPI infrastructure
message, stdout from the user process, stderr from the user process,
etc.).
Filtering is not active by default. Filter components must be
specifically requested, such as:
{{{
$ mpirun --mca filter xml ...
}}}
There can only be one filter component active.
= New MCA Parameters =
The new functionality described above introduces two new MCA
parameters:
* '''orte_base_help_aggregate''': Defaults to 1 (true), meaning that
help messages will be aggregated, as described above. If set to 0,
all help messages will be displayed, even if they are duplicates
(i.e., the original behavior).
* '''orte_base_show_output_recursions''': An MCA parameter to help
debug one of the known issues, described below. It is likely that
this MCA parameter will disappear before v1.3 final.
= Known Issues =
* The XML filter component is not complete. The current output from
this component is preliminary and not real XML. A bit more work
needs to be done to configure.m4 search for an appropriate XML
library/link it in/use it at run time.
* There are possible recursion loops in the orte_output() and
orte_show_help() functions -- e.g., if RML send calls orte_output()
or orte_show_help(). We have some ideas how to fix these, but
figured that it was ok to commit before feature freeze with known
issues. The code currently contains sub-optimal workarounds so
that this will not be a problem, but it would be good to actually
solve the problem rather than have hackish workarounds before v1.3 final.
This commit was SVN r18434.
2008-05-14 00:00:55 +04:00
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
#include "orte/mca/errmgr/errmgr.h"
|
2010-04-23 08:44:41 +04:00
|
|
|
#include "orte/mca/rml/rml.h"
|
2009-05-04 15:07:40 +04:00
|
|
|
#include "orte/util/proc_info.h"
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
#include "orte/util/name_fns.h"
|
2008-02-28 04:57:57 +03:00
|
|
|
|
|
|
|
|
#include "orte/runtime/runtime.h"
|
2008-06-18 07:15:56 +04:00
|
|
|
#include "orte/runtime/runtime_internals.h"
|
2008-02-28 04:57:57 +03:00
|
|
|
#include "orte/runtime/orte_globals.h"
|
|
|
|
|
|
|
|
|
|
/* need the data type support functions here */
|
|
|
|
|
#include "orte/runtime/data_type_support/orte_dt_support.h"
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
#if !ORTE_DISABLE_FULL_SUPPORT
|
2008-02-28 04:57:57 +03:00
|
|
|
|
|
|
|
|
/* globals used by RTE */
|
|
|
|
|
bool orte_timing;
|
2009-01-08 17:25:56 +03:00
|
|
|
FILE *orte_timing_output = NULL;
|
2009-01-12 22:12:58 +03:00
|
|
|
bool orte_timing_details;
|
2008-03-28 05:20:37 +03:00
|
|
|
bool orte_debug_daemons_file_flag = false;
|
2008-08-14 22:59:01 +04:00
|
|
|
bool orte_leave_session_attached;
|
2008-04-17 17:50:59 +04:00
|
|
|
bool orte_do_not_launch = false;
|
2008-03-28 05:20:37 +03:00
|
|
|
bool orted_spin_flag = false;
|
2009-09-22 06:16:40 +04:00
|
|
|
bool orte_daemon_bootstrap = false;
|
2010-08-09 23:28:56 +04:00
|
|
|
char *orte_local_cpu_type = NULL;
|
2009-12-01 02:11:25 +03:00
|
|
|
char *orte_local_cpu_model = NULL;
|
2010-07-18 01:03:27 +04:00
|
|
|
char *orte_basename = NULL;
|
2009-08-21 22:03:34 +04:00
|
|
|
|
|
|
|
|
/* ORTE OOB port flags */
|
2008-03-28 05:20:37 +03:00
|
|
|
bool orte_static_ports = false;
|
2009-08-21 22:03:34 +04:00
|
|
|
char *orte_oob_static_ports = NULL;
|
2009-08-22 06:58:20 +04:00
|
|
|
bool orte_standalone_operation = false;
|
2009-08-21 22:03:34 +04:00
|
|
|
|
2008-04-02 00:32:17 +04:00
|
|
|
bool orte_keep_fqdn_hostnames = false;
|
2008-11-24 22:57:08 +03:00
|
|
|
bool orte_show_resolved_nodenames;
|
2008-06-03 01:46:34 +04:00
|
|
|
int orted_debug_failure;
|
|
|
|
|
int orted_debug_failure_delay;
|
2008-06-24 21:50:56 +04:00
|
|
|
bool orte_homogeneous_nodes = false;
|
|
|
|
|
bool orte_hetero_apps = false;
|
2008-08-19 19:19:30 +04:00
|
|
|
bool orte_never_launched = false;
|
2008-09-23 19:46:34 +04:00
|
|
|
bool orte_devel_level_output = false;
|
2008-04-10 02:10:53 +04:00
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
char **orte_launch_environ;
|
2008-07-25 21:13:22 +04:00
|
|
|
|
|
|
|
|
bool orte_hnp_is_allocated = false;
|
2008-08-04 18:25:19 +04:00
|
|
|
bool orte_allocation_required;
|
2008-07-25 21:13:22 +04:00
|
|
|
|
2009-02-13 07:14:10 +03:00
|
|
|
char *orte_launch_agent = NULL;
|
2008-02-28 04:57:57 +03:00
|
|
|
char **orted_cmd_line=NULL;
|
2008-08-05 19:09:29 +04:00
|
|
|
|
2010-10-23 00:07:24 +04:00
|
|
|
/* debugger job */
|
2008-08-13 21:47:24 +04:00
|
|
|
orte_job_t *orte_debugger_daemon=NULL;
|
|
|
|
|
|
2010-07-18 01:03:27 +04:00
|
|
|
/* exit flags */
|
2008-02-28 04:57:57 +03:00
|
|
|
int orte_exit_status = 0;
|
|
|
|
|
bool orte_abnormal_term_ordered = false;
|
2008-11-01 00:10:00 +03:00
|
|
|
bool orte_routing_is_enabled = false;
|
2009-02-27 13:16:25 +03:00
|
|
|
bool orte_job_term_ordered = false;
|
2010-05-23 06:57:03 +04:00
|
|
|
bool orte_orteds_term_ordered = false;
|
2008-06-03 01:46:34 +04:00
|
|
|
|
|
|
|
|
int orte_startup_timeout;
|
2008-04-10 02:10:53 +04:00
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
int orte_timeout_usec_per_proc;
|
|
|
|
|
float orte_max_timeout;
|
|
|
|
|
|
2008-05-01 23:19:34 +04:00
|
|
|
opal_buffer_t *orte_tree_launch_cmd = NULL;
|
|
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
/* global arrays for data storage */
|
2008-02-28 08:32:23 +03:00
|
|
|
opal_pointer_array_t *orte_job_data;
|
|
|
|
|
opal_pointer_array_t *orte_node_pool;
|
2008-02-28 04:57:57 +03:00
|
|
|
|
2008-12-08 22:00:36 +03:00
|
|
|
/* a clean output channel without prefix */
|
|
|
|
|
int orte_clean_output = -1;
|
|
|
|
|
|
2009-01-07 17:58:38 +03:00
|
|
|
/* RHC: FLAG TO SELECT WHETHER OR NOT TO SEND PROFILE FILE IN NIDMAP */
|
|
|
|
|
bool orte_send_profile;
|
|
|
|
|
|
|
|
|
|
/* Nidmap and job maps */
|
|
|
|
|
opal_pointer_array_t orte_nidmap;
|
|
|
|
|
opal_pointer_array_t orte_jobmap;
|
2009-06-24 00:25:38 +04:00
|
|
|
bool orte_use_regexp;
|
|
|
|
|
char *orted_launch_cmd = NULL;
|
2009-01-07 17:58:38 +03:00
|
|
|
|
2009-01-08 17:25:56 +03:00
|
|
|
/* list of local children on a daemon */
|
|
|
|
|
opal_list_t orte_local_children;
|
2010-04-23 08:44:41 +04:00
|
|
|
opal_mutex_t orte_local_children_lock;
|
|
|
|
|
opal_condition_t orte_local_children_cond;
|
|
|
|
|
|
2009-01-08 17:25:56 +03:00
|
|
|
/* list of job data for local children on a daemon */
|
|
|
|
|
opal_list_t orte_local_jobdata;
|
2010-04-23 08:44:41 +04:00
|
|
|
opal_mutex_t orte_local_jobdata_lock;
|
|
|
|
|
opal_condition_t orte_local_jobdata_cond;
|
2009-01-08 17:25:56 +03:00
|
|
|
|
2009-01-31 01:47:30 +03:00
|
|
|
/* IOF controls */
|
|
|
|
|
bool orte_tag_output;
|
|
|
|
|
bool orte_timestamp_output;
|
|
|
|
|
char *orte_output_filename;
|
|
|
|
|
/* generate new xterm windows to display output from specified ranks */
|
|
|
|
|
char *orte_xterm;
|
|
|
|
|
|
2009-01-30 21:50:10 +03:00
|
|
|
/* whether or not to forward SIGTSTP and SIGCONT signals */
|
|
|
|
|
bool orte_forward_job_control;
|
|
|
|
|
|
2009-02-09 23:44:44 +03:00
|
|
|
/* rsh support */
|
2009-02-13 07:14:10 +03:00
|
|
|
char *orte_rsh_agent = NULL;
|
2009-05-30 05:10:25 +04:00
|
|
|
bool orte_assume_same_shell = true;
|
2009-02-09 23:44:44 +03:00
|
|
|
|
2009-06-03 03:52:02 +04:00
|
|
|
/* report launch progress */
|
|
|
|
|
bool orte_report_launch_progress = false;
|
|
|
|
|
|
2009-08-11 06:51:27 +04:00
|
|
|
/* cluster hardware info */
|
|
|
|
|
uint8_t orte_default_num_boards;
|
|
|
|
|
uint8_t orte_default_num_sockets_per_board;
|
|
|
|
|
uint8_t orte_default_num_cores_per_socket;
|
|
|
|
|
|
|
|
|
|
/* allocation specification */
|
|
|
|
|
char *orte_default_cpu_set;
|
2009-08-13 20:08:43 +04:00
|
|
|
char *orte_default_hostfile = NULL;
|
|
|
|
|
char *orte_rankfile;
|
2010-02-24 11:50:03 +03:00
|
|
|
#ifdef __WINDOWS__
|
2010-02-23 22:42:51 +03:00
|
|
|
char *orte_ccp_headnode;
|
2010-02-24 11:50:03 +03:00
|
|
|
#endif
|
2009-08-11 06:51:27 +04:00
|
|
|
|
|
|
|
|
/* default rank assigment and binding policy */
|
|
|
|
|
orte_mapping_policy_t orte_default_mapping_policy = 0;
|
|
|
|
|
|
2009-09-09 09:28:45 +04:00
|
|
|
/* tool communication controls */
|
|
|
|
|
bool orte_report_events = false;
|
|
|
|
|
char *orte_report_events_uri = NULL;
|
|
|
|
|
|
2009-09-28 07:17:15 +04:00
|
|
|
/* report bindings */
|
|
|
|
|
bool orte_report_bindings = false;
|
|
|
|
|
|
2010-01-14 20:59:42 +03:00
|
|
|
/* barrier control */
|
|
|
|
|
bool orte_do_not_barrier = false;
|
|
|
|
|
|
2010-04-28 08:06:57 +04:00
|
|
|
/* process recovery */
|
|
|
|
|
bool orte_enable_recovery;
|
|
|
|
|
int32_t orte_max_global_restarts;
|
|
|
|
|
int32_t orte_max_local_restarts;
|
|
|
|
|
|
2010-04-23 08:44:41 +04:00
|
|
|
/* comm fn for updating state */
|
|
|
|
|
orte_default_comm_fn_t orte_comm;
|
|
|
|
|
|
2010-05-12 22:11:58 +04:00
|
|
|
/* exit status reporting */
|
|
|
|
|
bool orte_report_child_jobs_separately;
|
|
|
|
|
struct timeval orte_child_time_to_exit;
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
#endif /* !ORTE_DISABLE_FULL_RTE */
|
|
|
|
|
|
2008-09-02 19:07:48 +04:00
|
|
|
int orte_debug_output = -1;
|
2008-09-01 21:15:01 +04:00
|
|
|
bool orte_debug_daemons_flag = false;
|
|
|
|
|
bool orte_xml_output = false;
|
2009-09-02 22:03:10 +04:00
|
|
|
FILE *orte_xml_fp = NULL;
|
2009-07-15 23:43:26 +04:00
|
|
|
char *orte_job_ident = NULL;
|
2010-04-02 18:19:38 +04:00
|
|
|
bool orte_execute_quiet = false;
|
2010-10-16 07:29:47 +04:00
|
|
|
bool orte_report_silent_errors = false;
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2008-08-01 02:11:46 +04:00
|
|
|
/* See comment in orte/tools/orterun/debuggers.c about this MCA
|
|
|
|
|
param */
|
|
|
|
|
bool orte_in_parallel_debugger = false;
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2008-05-28 17:29:58 +04:00
|
|
|
int orte_dt_init(void)
|
2008-02-28 04:57:57 +03:00
|
|
|
{
|
2008-05-28 17:29:58 +04:00
|
|
|
int rc;
|
|
|
|
|
opal_data_type_t tmp;
|
|
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
/* set default output */
|
2008-06-09 18:53:58 +04:00
|
|
|
orte_debug_output = opal_output_open(NULL);
|
2008-12-08 22:00:36 +03:00
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
/* open up the verbose output for ORTE debugging */
|
|
|
|
|
if (orte_debug_flag || 0 < orte_debug_verbosity ||
|
2009-05-04 15:07:40 +04:00
|
|
|
(orte_debug_daemons_flag && (ORTE_PROC_IS_DAEMON || ORTE_PROC_IS_HNP))) {
|
2008-02-28 04:57:57 +03:00
|
|
|
if (0 < orte_debug_verbosity) {
|
2008-06-09 18:53:58 +04:00
|
|
|
opal_output_set_verbosity(orte_debug_output, orte_debug_verbosity);
|
2008-02-28 04:57:57 +03:00
|
|
|
} else {
|
2008-06-09 18:53:58 +04:00
|
|
|
opal_output_set_verbosity(orte_debug_output, 1);
|
2008-02-28 04:57:57 +03:00
|
|
|
}
|
|
|
|
|
}
|
2008-05-28 17:29:58 +04:00
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
/** register the base system types with the DSS */
|
|
|
|
|
tmp = ORTE_STD_CNTR;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_std_cntr,
|
|
|
|
|
orte_dt_unpack_std_cntr,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_std_cntr,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_std_cntr,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_STD_CNTR", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
tmp = ORTE_NAME;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_name,
|
|
|
|
|
orte_dt_unpack_name,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_name,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_name,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_name,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_NAME", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_VPID;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_vpid,
|
|
|
|
|
orte_dt_unpack_vpid,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_vpid,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_vpid,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_VPID", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_JOBID;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_jobid,
|
|
|
|
|
orte_dt_unpack_jobid,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_jobid,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_jobid,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_JOBID", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
|
|
|
|
#if !ORTE_DISABLE_FULL_SUPPORT
|
2008-02-28 04:57:57 +03:00
|
|
|
tmp = ORTE_JOB;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_job,
|
|
|
|
|
orte_dt_unpack_job,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_job,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_job,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_size_job,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_job,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_obj_release,
|
|
|
|
|
OPAL_DSS_STRUCTURED,
|
|
|
|
|
"ORTE_JOB", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_NODE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_node,
|
|
|
|
|
orte_dt_unpack_node,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_node,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_node,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_size_node,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_node,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_obj_release,
|
|
|
|
|
OPAL_DSS_STRUCTURED,
|
|
|
|
|
"ORTE_NODE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_PROC;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_proc,
|
|
|
|
|
orte_dt_unpack_proc,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_proc,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_proc,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_size_proc,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_proc,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_obj_release,
|
|
|
|
|
OPAL_DSS_STRUCTURED,
|
|
|
|
|
"ORTE_PROC", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_APP_CONTEXT;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_app_context,
|
|
|
|
|
orte_dt_unpack_app_context,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_app_context,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_app_context,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_size_app_context,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_app_context,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_obj_release,
|
|
|
|
|
OPAL_DSS_STRUCTURED,
|
|
|
|
|
"ORTE_APP_CONTEXT", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_NODE_STATE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_node_state,
|
|
|
|
|
orte_dt_unpack_node_state,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_node_state,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_node_state,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_NODE_STATE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_PROC_STATE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_proc_state,
|
|
|
|
|
orte_dt_unpack_proc_state,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_proc_state,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_proc_state,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_PROC_STATE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_JOB_STATE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_job_state,
|
|
|
|
|
orte_dt_unpack_job_state,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_job_state,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_job_state,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_JOB_STATE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_EXIT_CODE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_exit_code,
|
|
|
|
|
orte_dt_unpack_exit_code,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_exit_code,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_exit_code,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_EXIT_CODE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_JOB_MAP;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_map,
|
|
|
|
|
orte_dt_unpack_map,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_map,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_map,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_size_map,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_print_map,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_obj_release,
|
|
|
|
|
OPAL_DSS_STRUCTURED,
|
|
|
|
|
"ORTE_JOB_MAP", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_RML_TAG;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_tag,
|
|
|
|
|
orte_dt_unpack_tag,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_tag,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_tags,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_RML_TAG", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_DAEMON_CMD;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_daemon_cmd,
|
|
|
|
|
orte_dt_unpack_daemon_cmd,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_daemon_cmd,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_daemon_cmd,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_DAEMON_CMD", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
tmp = ORTE_GRPCOMM_MODE;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_grpcomm_mode,
|
|
|
|
|
orte_dt_unpack_grpcomm_mode,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_grpcomm_mode,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_grpcomm_mode,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_GRPCOMM_MODE", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
Roll in the revamped IOF subsystem. Per the devel mailing list email, this is a complete rewrite of the iof framework designed to simplify the code for maintainability, and to support features we had planned to do, but were too difficult to implement in the old code. Specifically, the new code:
1. completely and cleanly separates responsibilities between the HNP, orted, and tool components.
2. removes all wireup messaging during launch and shutdown.
3. maintains flow control for stdin to avoid large-scale consumption of memory by orteds when large input files are forwarded. This is done using an xon/xoff protocol.
4. enables specification of stdin recipients on the mpirun cmd line. Allowed options include rank, "all", or "none". Default is rank 0.
5. creates a new MPI_Info key "ompi_stdin_target" that supports the above options for child jobs. Default is "none".
6. adds a new tool "orte-iof" that can connect to a running mpirun and display the output. Cmd line options allow selection of any combination of stdout, stderr, and stddiag. Default is stdout.
7. adds a new mpirun and orte-iof cmd line option "tag-output" that will tag each line of output with process name and stream ident. For example, "[1,0]<stdout>this is output"
This is not intended for the 1.3 release as it is a major change requiring considerable soak time.
This commit was SVN r19767.
2008-10-18 04:00:49 +04:00
|
|
|
tmp = ORTE_IOF_TAG;
|
|
|
|
|
if (ORTE_SUCCESS != (rc = opal_dss.register_type(orte_dt_pack_iof_tag,
|
|
|
|
|
orte_dt_unpack_iof_tag,
|
|
|
|
|
(opal_dss_copy_fn_t)orte_dt_copy_iof_tag,
|
|
|
|
|
(opal_dss_compare_fn_t)orte_dt_compare_iof_tag,
|
|
|
|
|
(opal_dss_size_fn_t)orte_dt_std_size,
|
|
|
|
|
(opal_dss_print_fn_t)orte_dt_std_print,
|
|
|
|
|
(opal_dss_release_fn_t)orte_dt_std_release,
|
|
|
|
|
OPAL_DSS_UNSTRUCTURED,
|
|
|
|
|
"ORTE_IOF_TAG", &tmp))) {
|
|
|
|
|
ORTE_ERROR_LOG(rc);
|
|
|
|
|
return rc;
|
|
|
|
|
}
|
2008-12-16 23:09:56 +03:00
|
|
|
#endif /* !ORTE_DISABLE_FULL_SUPPORT */
|
|
|
|
|
|
2008-02-28 04:57:57 +03:00
|
|
|
return ORTE_SUCCESS;
|
|
|
|
|
}
|
|
|
|
|
|
2008-08-31 22:06:55 +04:00
|
|
|
#if !ORTE_DISABLE_FULL_SUPPORT
|
2008-02-28 04:57:57 +03:00
|
|
|
|
|
|
|
|
orte_job_t* orte_get_job_data_object(orte_jobid_t job)
|
|
|
|
|
{
|
2009-04-13 23:06:54 +04:00
|
|
|
int32_t ljob;
|
2008-02-28 04:57:57 +03:00
|
|
|
|
2010-03-31 23:20:06 +04:00
|
|
|
/* if the job data wasn't setup, we cannot provide the data */
|
|
|
|
|
if (NULL == orte_job_data) {
|
2008-02-28 04:57:57 +03:00
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
2009-04-13 23:06:54 +04:00
|
|
|
/* the job is indexed by its local jobid, so we can
|
|
|
|
|
* just look it up here. it is not an error for this
|
|
|
|
|
* to not be found - could just be
|
2009-03-24 21:06:49 +03:00
|
|
|
* a race condition whereby the job has already been
|
2009-04-13 23:06:54 +04:00
|
|
|
* removed from the array. The get_item function
|
|
|
|
|
* will just return NULL in that case.
|
2009-03-24 21:06:49 +03:00
|
|
|
*/
|
2009-04-13 23:06:54 +04:00
|
|
|
ljob = ORTE_LOCAL_JOBID(job);
|
|
|
|
|
return (orte_job_t*)opal_pointer_array_get_item(orte_job_data, ljob);
|
2008-02-28 04:57:57 +03:00
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2010-04-23 08:44:41 +04:00
|
|
|
int orte_global_comm(orte_process_name_t *recipient,
|
|
|
|
|
opal_buffer_t *buf, orte_rml_tag_t tag,
|
|
|
|
|
orte_default_cbfunc_t cbfunc)
|
|
|
|
|
{
|
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
|
|
if (recipient->jobid == ORTE_PROC_MY_NAME->jobid &&
|
|
|
|
|
recipient->vpid == ORTE_PROC_MY_NAME->vpid &&
|
|
|
|
|
NULL != cbfunc) {
|
|
|
|
|
/* if I am the recipient and a direct fn is provided, use a message event */
|
|
|
|
|
ORTE_MESSAGE_EVENT(ORTE_PROC_MY_NAME, buf, tag, cbfunc);
|
|
|
|
|
ret = ORTE_SUCCESS;
|
|
|
|
|
} else {
|
|
|
|
|
/* go ahead and send it */
|
|
|
|
|
if (0 > (ret = orte_rml.send_buffer(recipient, buf, tag, 0))) {
|
|
|
|
|
ORTE_ERROR_LOG(ret);
|
|
|
|
|
} else {
|
|
|
|
|
ret = ORTE_SUCCESS;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return ret;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* CONSTRUCTORS, DESTRUCTORS, AND CLASS INSTANTIATIONS
|
|
|
|
|
* FOR ORTE CLASSES
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
static void orte_app_context_construct(orte_app_context_t* app_context)
|
|
|
|
|
{
|
2010-06-07 18:21:08 +04:00
|
|
|
app_context->name = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
app_context->idx=0;
|
|
|
|
|
app_context->app=NULL;
|
|
|
|
|
app_context->num_procs=0;
|
|
|
|
|
app_context->argv=NULL;
|
|
|
|
|
app_context->env=NULL;
|
|
|
|
|
app_context->cwd=NULL;
|
|
|
|
|
app_context->user_specified_cwd=false;
|
|
|
|
|
app_context->hostfile=NULL;
|
|
|
|
|
app_context->add_hostfile=NULL;
|
2009-07-14 18:34:11 +04:00
|
|
|
app_context->add_host = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
app_context->dash_host = NULL;
|
|
|
|
|
app_context->prefix_dir = NULL;
|
|
|
|
|
app_context->preload_binary = false;
|
|
|
|
|
app_context->preload_files = NULL;
|
|
|
|
|
app_context->preload_files_dest_dir = NULL;
|
2009-02-12 18:56:45 +03:00
|
|
|
app_context->preload_files_src_dir = NULL;
|
2009-01-25 15:39:24 +03:00
|
|
|
app_context->used_on_node = false;
|
A number of C/R enhancements per RFC below:
http://www.open-mpi.org/community/lists/devel/2010/07/8240.php
Documentation:
http://osl.iu.edu/research/ft/
Major Changes:
--------------
* Added C/R-enabled Debugging support.
Enabled with the --enable-crdebug flag. See the following website for more information:
http://osl.iu.edu/research/ft/crdebug/
* Added Stable Storage (SStore) framework for checkpoint storage
* 'central' component does a direct to central storage save
* 'stage' component stages checkpoints to central storage while the application continues execution.
* 'stage' supports offline compression of checkpoints before moving (sstore_stage_compress)
* 'stage' supports local caching of checkpoints to improve automatic recovery (sstore_stage_caching)
* Added Compression (compress) framework to support
* Add two new ErrMgr recovery policies
* {{{crmig}}} C/R Process Migration
* {{{autor}}} C/R Automatic Recovery
* Added the {{{ompi-migrate}}} command line tool to support the {{{crmig}}} ErrMgr component
* Added CR MPI Ext functions (enable them with {{{--enable-mpi-ext=cr}}} configure option)
* {{{OMPI_CR_Checkpoint}}} (Fixes trac:2342)
* {{{OMPI_CR_Restart}}}
* {{{OMPI_CR_Migrate}}} (may need some more work for mapping rules)
* {{{OMPI_CR_INC_register_callback}}} (Fixes trac:2192)
* {{{OMPI_CR_Quiesce_start}}}
* {{{OMPI_CR_Quiesce_checkpoint}}}
* {{{OMPI_CR_Quiesce_end}}}
* {{{OMPI_CR_self_register_checkpoint_callback}}}
* {{{OMPI_CR_self_register_restart_callback}}}
* {{{OMPI_CR_self_register_continue_callback}}}
* The ErrMgr predicted_fault() interface has been changed to take an opal_list_t of ErrMgr defined types. This will allow us to better support a wider range of fault prediction services in the future.
* Add a progress meter to:
* FileM rsh (filem_rsh_process_meter)
* SnapC full (snapc_full_progress_meter)
* SStore stage (sstore_stage_progress_meter)
* Added 2 new command line options to ompi-restart
* --showme : Display the full command line that would have been exec'ed.
* --mpirun_opts : Command line options to pass directly to mpirun. (Fixes trac:2413)
* Deprecated some MCA params:
* crs_base_snapshot_dir deprecated, use sstore_stage_local_snapshot_dir
* snapc_base_global_snapshot_dir deprecated, use sstore_base_global_snapshot_dir
* snapc_base_global_shared deprecated, use sstore_stage_global_is_shared
* snapc_base_store_in_place deprecated, replaced with different components of SStore
* snapc_base_global_snapshot_ref deprecated, use sstore_base_global_snapshot_ref
* snapc_base_establish_global_snapshot_dir deprecated, never well supported
* snapc_full_skip_filem deprecated, use sstore_stage_skip_filem
Minor Changes:
--------------
* Fixes trac:1924 : {{{ompi-restart}}} now recognizes path prefixed checkpoint handles and does the right thing.
* Fixes trac:2097 : {{{ompi-info}}} should now report all available CRS components
* Fixes trac:2161 : Manual checkpoint movement. A user can 'mv' a checkpoint directory from the original location to another and still restart from it.
* Fixes trac:2208 : Honor various TMPDIR varaibles instead of forcing {{{/tmp}}}
* Move {{{ompi_cr_continue_like_restart}}} to {{{orte_cr_continue_like_restart}}} to be more flexible in where this should be set.
* opal_crs_base_metadata_write* functions have been moved to SStore to support a wider range of metadata handling functionality.
* Cleanup the CRS framework and components to work with the SStore framework.
* Cleanup the SnapC framework and components to work with the SStore framework (cleans up these code paths considerably).
* Add 'quiesce' hook to CRCP for a future enhancement.
* We now require a BLCR version that supports {{{cr_request_file()}}} or {{{cr_request_checkpoint()}}} in order to make the code more maintainable. Note that {{{cr_request_file}}} has been deprecated since 0.7.0, so we prefer to use {{{cr_request_checkpoint()}}}.
* Add optional application level INC callbacks (registered through the CR MPI Ext interface).
* Increase the {{{opal_cr_thread_sleep_wait}}} parameter to 1000 microseconds to make the C/R thread less aggressive.
* {{{opal-restart}}} now looks for cache directories before falling back on stable storage when asked.
* {{{opal-restart}}} also support local decompression before restarting
* {{{orte-checkpoint}}} now uses the SStore framework to work with the metadata
* {{{orte-restart}}} now uses the SStore framework to work with the metadata
* Remove the {{{orte-restart}}} preload option. This was removed since the user only needs to select the 'stage' component in order to support this functionality.
* Since the '-am' parameter is saved in the metadata, {{{ompi-restart}}} no longer hard codes {{{-am ft-enable-cr}}}.
* Fix {{{hnp}}} ErrMgr so that if a previous component in the stack has 'fixed' the problem, then it should be skipped.
* Make sure to decrement the number of 'num_local_procs' in the orted when one goes away.
* odls now checks the SStore framework to see if it needs to load any checkpoint files before launching (to support 'stage'). This separates the SStore logic from the --preload-[binary|files] options.
* Add unique IDs to the named pipes established between the orted and the app in SnapC. This is to better support migration and automatic recovery activities.
* Improve the checks for 'already checkpointing' error path.
* A a recovery output timer, to show how long it takes to restart a job
* Do a better job of cleaning up the old session directory on restart.
* Add a local module to the autor and crmig ErrMgr components. These small modules prevent the 'orted' component from attempting a local recovery (Which does not work for MPI apps at the moment)
* Add a fix for bounding the checkpointable region between MPI_Init and MPI_Finalize.
This commit was SVN r23587.
The following Trac tickets were found above:
Ticket 1924 --> https://svn.open-mpi.org/trac/ompi/ticket/1924
Ticket 2097 --> https://svn.open-mpi.org/trac/ompi/ticket/2097
Ticket 2161 --> https://svn.open-mpi.org/trac/ompi/ticket/2161
Ticket 2192 --> https://svn.open-mpi.org/trac/ompi/ticket/2192
Ticket 2208 --> https://svn.open-mpi.org/trac/ompi/ticket/2208
Ticket 2342 --> https://svn.open-mpi.org/trac/ompi/ticket/2342
Ticket 2413 --> https://svn.open-mpi.org/trac/ompi/ticket/2413
2010-08-11 00:51:11 +04:00
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
|
|
|
|
app_context->sstore_load = NULL;
|
|
|
|
|
#endif
|
2010-06-07 18:21:08 +04:00
|
|
|
app_context->max_local_restarts = -1;
|
|
|
|
|
app_context->max_global_restarts = -1;
|
2010-07-18 01:03:27 +04:00
|
|
|
app_context->constrain = true;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_app_context_destructor(orte_app_context_t* app_context)
|
|
|
|
|
{
|
2010-06-07 18:21:08 +04:00
|
|
|
if (NULL != app_context->name) {
|
|
|
|
|
free(app_context->name);
|
|
|
|
|
}
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != app_context->app) {
|
|
|
|
|
free (app_context->app);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->app = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* argv and env lists created by util/argv copy functions */
|
|
|
|
|
if (NULL != app_context->argv) {
|
|
|
|
|
opal_argv_free(app_context->argv);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->argv = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != app_context->env) {
|
|
|
|
|
opal_argv_free(app_context->env);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->env = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != app_context->cwd) {
|
|
|
|
|
free (app_context->cwd);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->cwd = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != app_context->hostfile) {
|
|
|
|
|
free(app_context->hostfile);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->hostfile = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != app_context->add_hostfile) {
|
|
|
|
|
free(app_context->add_hostfile);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->add_hostfile = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
2009-07-14 18:34:11 +04:00
|
|
|
if (NULL != app_context->add_host) {
|
|
|
|
|
opal_argv_free(app_context->add_host);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->add_host = NULL;
|
2009-07-14 18:34:11 +04:00
|
|
|
}
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != app_context->dash_host) {
|
2009-07-14 22:56:49 +04:00
|
|
|
opal_argv_free(app_context->dash_host);
|
|
|
|
|
app_context->dash_host = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != app_context->prefix_dir) {
|
|
|
|
|
free(app_context->prefix_dir);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->prefix_dir = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
app_context->preload_binary = false;
|
2009-06-12 21:53:13 +04:00
|
|
|
app_context->preload_libs = false;
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
if(NULL != app_context->preload_files) {
|
|
|
|
|
free(app_context->preload_files);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->preload_files = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if(NULL != app_context->preload_files_dest_dir) {
|
|
|
|
|
free(app_context->preload_files_dest_dir);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->preload_files_dest_dir = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
2009-02-12 18:56:45 +03:00
|
|
|
|
|
|
|
|
if(NULL != app_context->preload_files_src_dir) {
|
|
|
|
|
free(app_context->preload_files_src_dir);
|
2009-07-14 22:56:49 +04:00
|
|
|
app_context->preload_files_src_dir = NULL;
|
2009-02-12 18:56:45 +03:00
|
|
|
}
|
A number of C/R enhancements per RFC below:
http://www.open-mpi.org/community/lists/devel/2010/07/8240.php
Documentation:
http://osl.iu.edu/research/ft/
Major Changes:
--------------
* Added C/R-enabled Debugging support.
Enabled with the --enable-crdebug flag. See the following website for more information:
http://osl.iu.edu/research/ft/crdebug/
* Added Stable Storage (SStore) framework for checkpoint storage
* 'central' component does a direct to central storage save
* 'stage' component stages checkpoints to central storage while the application continues execution.
* 'stage' supports offline compression of checkpoints before moving (sstore_stage_compress)
* 'stage' supports local caching of checkpoints to improve automatic recovery (sstore_stage_caching)
* Added Compression (compress) framework to support
* Add two new ErrMgr recovery policies
* {{{crmig}}} C/R Process Migration
* {{{autor}}} C/R Automatic Recovery
* Added the {{{ompi-migrate}}} command line tool to support the {{{crmig}}} ErrMgr component
* Added CR MPI Ext functions (enable them with {{{--enable-mpi-ext=cr}}} configure option)
* {{{OMPI_CR_Checkpoint}}} (Fixes trac:2342)
* {{{OMPI_CR_Restart}}}
* {{{OMPI_CR_Migrate}}} (may need some more work for mapping rules)
* {{{OMPI_CR_INC_register_callback}}} (Fixes trac:2192)
* {{{OMPI_CR_Quiesce_start}}}
* {{{OMPI_CR_Quiesce_checkpoint}}}
* {{{OMPI_CR_Quiesce_end}}}
* {{{OMPI_CR_self_register_checkpoint_callback}}}
* {{{OMPI_CR_self_register_restart_callback}}}
* {{{OMPI_CR_self_register_continue_callback}}}
* The ErrMgr predicted_fault() interface has been changed to take an opal_list_t of ErrMgr defined types. This will allow us to better support a wider range of fault prediction services in the future.
* Add a progress meter to:
* FileM rsh (filem_rsh_process_meter)
* SnapC full (snapc_full_progress_meter)
* SStore stage (sstore_stage_progress_meter)
* Added 2 new command line options to ompi-restart
* --showme : Display the full command line that would have been exec'ed.
* --mpirun_opts : Command line options to pass directly to mpirun. (Fixes trac:2413)
* Deprecated some MCA params:
* crs_base_snapshot_dir deprecated, use sstore_stage_local_snapshot_dir
* snapc_base_global_snapshot_dir deprecated, use sstore_base_global_snapshot_dir
* snapc_base_global_shared deprecated, use sstore_stage_global_is_shared
* snapc_base_store_in_place deprecated, replaced with different components of SStore
* snapc_base_global_snapshot_ref deprecated, use sstore_base_global_snapshot_ref
* snapc_base_establish_global_snapshot_dir deprecated, never well supported
* snapc_full_skip_filem deprecated, use sstore_stage_skip_filem
Minor Changes:
--------------
* Fixes trac:1924 : {{{ompi-restart}}} now recognizes path prefixed checkpoint handles and does the right thing.
* Fixes trac:2097 : {{{ompi-info}}} should now report all available CRS components
* Fixes trac:2161 : Manual checkpoint movement. A user can 'mv' a checkpoint directory from the original location to another and still restart from it.
* Fixes trac:2208 : Honor various TMPDIR varaibles instead of forcing {{{/tmp}}}
* Move {{{ompi_cr_continue_like_restart}}} to {{{orte_cr_continue_like_restart}}} to be more flexible in where this should be set.
* opal_crs_base_metadata_write* functions have been moved to SStore to support a wider range of metadata handling functionality.
* Cleanup the CRS framework and components to work with the SStore framework.
* Cleanup the SnapC framework and components to work with the SStore framework (cleans up these code paths considerably).
* Add 'quiesce' hook to CRCP for a future enhancement.
* We now require a BLCR version that supports {{{cr_request_file()}}} or {{{cr_request_checkpoint()}}} in order to make the code more maintainable. Note that {{{cr_request_file}}} has been deprecated since 0.7.0, so we prefer to use {{{cr_request_checkpoint()}}}.
* Add optional application level INC callbacks (registered through the CR MPI Ext interface).
* Increase the {{{opal_cr_thread_sleep_wait}}} parameter to 1000 microseconds to make the C/R thread less aggressive.
* {{{opal-restart}}} now looks for cache directories before falling back on stable storage when asked.
* {{{opal-restart}}} also support local decompression before restarting
* {{{orte-checkpoint}}} now uses the SStore framework to work with the metadata
* {{{orte-restart}}} now uses the SStore framework to work with the metadata
* Remove the {{{orte-restart}}} preload option. This was removed since the user only needs to select the 'stage' component in order to support this functionality.
* Since the '-am' parameter is saved in the metadata, {{{ompi-restart}}} no longer hard codes {{{-am ft-enable-cr}}}.
* Fix {{{hnp}}} ErrMgr so that if a previous component in the stack has 'fixed' the problem, then it should be skipped.
* Make sure to decrement the number of 'num_local_procs' in the orted when one goes away.
* odls now checks the SStore framework to see if it needs to load any checkpoint files before launching (to support 'stage'). This separates the SStore logic from the --preload-[binary|files] options.
* Add unique IDs to the named pipes established between the orted and the app in SnapC. This is to better support migration and automatic recovery activities.
* Improve the checks for 'already checkpointing' error path.
* A a recovery output timer, to show how long it takes to restart a job
* Do a better job of cleaning up the old session directory on restart.
* Add a local module to the autor and crmig ErrMgr components. These small modules prevent the 'orted' component from attempting a local recovery (Which does not work for MPI apps at the moment)
* Add a fix for bounding the checkpointable region between MPI_Init and MPI_Finalize.
This commit was SVN r23587.
The following Trac tickets were found above:
Ticket 1924 --> https://svn.open-mpi.org/trac/ompi/ticket/1924
Ticket 2097 --> https://svn.open-mpi.org/trac/ompi/ticket/2097
Ticket 2161 --> https://svn.open-mpi.org/trac/ompi/ticket/2161
Ticket 2192 --> https://svn.open-mpi.org/trac/ompi/ticket/2192
Ticket 2208 --> https://svn.open-mpi.org/trac/ompi/ticket/2208
Ticket 2342 --> https://svn.open-mpi.org/trac/ompi/ticket/2342
Ticket 2413 --> https://svn.open-mpi.org/trac/ompi/ticket/2413
2010-08-11 00:51:11 +04:00
|
|
|
|
|
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
|
|
|
|
if( NULL != app_context->sstore_load ) {
|
|
|
|
|
free(app_context->sstore_load);
|
|
|
|
|
app_context->sstore_load = NULL;
|
|
|
|
|
}
|
|
|
|
|
#endif
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_app_context_t,
|
|
|
|
|
opal_object_t,
|
|
|
|
|
orte_app_context_construct,
|
|
|
|
|
orte_app_context_destructor);
|
|
|
|
|
|
|
|
|
|
static void orte_job_construct(orte_job_t* job)
|
|
|
|
|
{
|
2010-06-07 18:21:08 +04:00
|
|
|
job->name = NULL;
|
2010-06-24 05:57:27 +04:00
|
|
|
job->instance = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
job->jobid = ORTE_JOBID_INVALID;
|
|
|
|
|
job->apps = OBJ_NEW(opal_pointer_array_t);
|
|
|
|
|
opal_pointer_array_init(job->apps,
|
|
|
|
|
1,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_MAX_SIZE,
|
|
|
|
|
2);
|
|
|
|
|
job->num_apps = 0;
|
2008-10-06 21:56:23 +04:00
|
|
|
job->controls = ORTE_JOB_CONTROL_FORWARD_OUTPUT;
|
Roll in the revamped IOF subsystem. Per the devel mailing list email, this is a complete rewrite of the iof framework designed to simplify the code for maintainability, and to support features we had planned to do, but were too difficult to implement in the old code. Specifically, the new code:
1. completely and cleanly separates responsibilities between the HNP, orted, and tool components.
2. removes all wireup messaging during launch and shutdown.
3. maintains flow control for stdin to avoid large-scale consumption of memory by orteds when large input files are forwarded. This is done using an xon/xoff protocol.
4. enables specification of stdin recipients on the mpirun cmd line. Allowed options include rank, "all", or "none". Default is rank 0.
5. creates a new MPI_Info key "ompi_stdin_target" that supports the above options for child jobs. Default is "none".
6. adds a new tool "orte-iof" that can connect to a running mpirun and display the output. Cmd line options allow selection of any combination of stdout, stderr, and stddiag. Default is stdout.
7. adds a new mpirun and orte-iof cmd line option "tag-output" that will tag each line of output with process name and stream ident. For example, "[1,0]<stdout>this is output"
This is not intended for the 1.3 release as it is a major change requiring considerable soak time.
This commit was SVN r19767.
2008-10-18 04:00:49 +04:00
|
|
|
job->stdin_target = ORTE_VPID_INVALID;
|
2008-09-01 21:15:01 +04:00
|
|
|
job->total_slots_alloc = 0;
|
|
|
|
|
job->num_procs = 0;
|
|
|
|
|
job->procs = OBJ_NEW(opal_pointer_array_t);
|
|
|
|
|
opal_pointer_array_init(job->procs,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_MAX_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE);
|
|
|
|
|
|
|
|
|
|
job->map = NULL;
|
|
|
|
|
job->bookmark = NULL;
|
|
|
|
|
job->oversubscribe_override = false;
|
|
|
|
|
job->state = ORTE_JOB_STATE_UNDEF;
|
|
|
|
|
|
|
|
|
|
job->num_launched = 0;
|
|
|
|
|
job->num_reported = 0;
|
|
|
|
|
job->num_terminated = 0;
|
2010-04-23 08:44:41 +04:00
|
|
|
job->num_daemons_reported = 0;
|
2008-09-01 21:15:01 +04:00
|
|
|
job->abort = false;
|
|
|
|
|
job->aborted_proc = NULL;
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
|
2010-08-28 18:03:17 +04:00
|
|
|
OBJ_CONSTRUCT(&job->dyn_spawn_lock, opal_mutex_t);
|
|
|
|
|
OBJ_CONSTRUCT(&job->dyn_spawn_cond, opal_condition_t);
|
|
|
|
|
job->dyn_spawn_active = false;
|
2010-04-23 08:44:41 +04:00
|
|
|
|
2010-04-28 08:06:57 +04:00
|
|
|
job->enable_recovery = false;
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
|
2010-04-23 08:44:41 +04:00
|
|
|
job->launch_msg_sent.tv_sec = 0;
|
|
|
|
|
job->launch_msg_sent.tv_usec = 0;
|
|
|
|
|
job->max_launch_msg_recvd.tv_sec = 0;
|
|
|
|
|
job->max_launch_msg_recvd.tv_usec = 0;
|
|
|
|
|
|
2010-03-13 02:57:50 +03:00
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
2008-09-01 21:15:01 +04:00
|
|
|
job->ckpt_state = 0;
|
|
|
|
|
job->ckpt_snapshot_ref = NULL;
|
|
|
|
|
job->ckpt_snapshot_loc = NULL;
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_job_destruct(orte_job_t* job)
|
|
|
|
|
{
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
orte_proc_t *proc;
|
|
|
|
|
orte_app_context_t *app;
|
|
|
|
|
orte_job_t *jdata;
|
2009-03-03 19:39:13 +03:00
|
|
|
int n;
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2009-05-11 18:03:07 +04:00
|
|
|
if (NULL == job) {
|
|
|
|
|
/* probably just a race condition - just return */
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (orte_debug_flag) {
|
|
|
|
|
opal_output(0, "%s Releasing job data for %s",
|
|
|
|
|
ORTE_NAME_PRINT(ORTE_PROC_MY_NAME), ORTE_JOBID_PRINT(job->jobid));
|
|
|
|
|
}
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
|
2010-06-24 05:57:27 +04:00
|
|
|
if (NULL != job->name) {
|
|
|
|
|
free(job->name);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (NULL != job->instance) {
|
|
|
|
|
free(job->instance);
|
|
|
|
|
}
|
|
|
|
|
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
for (n=0; n < job->apps->size; n++) {
|
|
|
|
|
if (NULL == (app = (orte_app_context_t*)opal_pointer_array_get_item(job->apps, n))) {
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
OBJ_RELEASE(app);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
OBJ_RELEASE(job->apps);
|
|
|
|
|
|
2009-07-14 22:56:49 +04:00
|
|
|
if (NULL != job->map) {
|
|
|
|
|
OBJ_RELEASE(job->map);
|
|
|
|
|
job->map = NULL;
|
|
|
|
|
}
|
2009-03-03 19:39:13 +03:00
|
|
|
|
|
|
|
|
for (n=0; n < job->procs->size; n++) {
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
if (NULL == (proc = (orte_proc_t*)opal_pointer_array_get_item(job->procs, n))) {
|
|
|
|
|
continue;
|
2009-03-03 19:39:13 +03:00
|
|
|
}
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
OBJ_RELEASE(proc);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
OBJ_RELEASE(job->procs);
|
|
|
|
|
|
2010-08-28 18:03:17 +04:00
|
|
|
OBJ_DESTRUCT(&job->dyn_spawn_lock);
|
|
|
|
|
OBJ_DESTRUCT(&job->dyn_spawn_cond);
|
2010-04-23 08:44:41 +04:00
|
|
|
|
2010-03-13 02:57:50 +03:00
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != job->ckpt_snapshot_ref) {
|
|
|
|
|
free(job->ckpt_snapshot_ref);
|
2009-07-14 22:56:49 +04:00
|
|
|
job->ckpt_snapshot_ref = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
if (NULL != job->ckpt_snapshot_loc) {
|
|
|
|
|
free(job->ckpt_snapshot_loc);
|
2009-07-14 22:56:49 +04:00
|
|
|
job->ckpt_snapshot_loc = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
#endif
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
|
|
|
|
|
/* find the job in the global array */
|
2009-05-11 18:03:07 +04:00
|
|
|
if (NULL != orte_job_data) {
|
|
|
|
|
for (n=0; n < orte_job_data->size; n++) {
|
|
|
|
|
if (NULL == (jdata = (orte_job_t*)opal_pointer_array_get_item(orte_job_data, n))) {
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
if (jdata->jobid == job->jobid) {
|
|
|
|
|
/* set the entry to NULL */
|
|
|
|
|
opal_pointer_array_set_item(orte_job_data, n, NULL);
|
|
|
|
|
break;
|
|
|
|
|
}
|
The current errmgr.register_callback API takes a jobid as one of its argument. The intent was to have the errmgr check the jobid of the job being reported to it and, if it matches the jobid that was registered, call the specified callback function.
Unfortunately, we assign the jobid during the plm.spawn procedure - which means it happens -after- control of the job has passed out of the range of mpirun (or whatever program is spawning the job), so it is too late for that main program to register a callback function. If the main program registers tha callback -after- we return from plm.spawn, then it (a) cannot get a callback for failed-to-start, and (b) will miss the callback if a proc aborts in the time between job launch and the call to errmgr.register_callback.
This commit fixes the problem by adding callback-related fields to the orte_job_t object. Thus, the main program can specify what job states should initiate a callback, what function is to be called, and what data is to be passed back by simply filling in the orte_job_t fields prior to calling plm.spawn.
Also, fully implement the "copy" function for the orte_job_t object.
NOTE: as a result of this change, the errmgr.register_callback API may no longer be of any value.
This commit was SVN r21200.
2009-05-11 07:38:15 +04:00
|
|
|
}
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_job_t,
|
|
|
|
|
opal_list_item_t,
|
|
|
|
|
orte_job_construct,
|
|
|
|
|
orte_job_destruct);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static void orte_node_construct(orte_node_t* node)
|
|
|
|
|
{
|
|
|
|
|
node->name = NULL;
|
2009-01-15 21:11:50 +03:00
|
|
|
node->alias = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
node->index = -1;
|
|
|
|
|
node->daemon = NULL;
|
|
|
|
|
node->daemon_launched = false;
|
|
|
|
|
node->launch_id = -1;
|
|
|
|
|
|
|
|
|
|
node->num_procs = 0;
|
|
|
|
|
node->procs = OBJ_NEW(opal_pointer_array_t);
|
|
|
|
|
opal_pointer_array_init(node->procs,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_MAX_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE);
|
|
|
|
|
node->next_node_rank = 0;
|
|
|
|
|
|
|
|
|
|
node->oversubscribed = false;
|
|
|
|
|
node->state = ORTE_NODE_STATE_UNKNOWN;
|
|
|
|
|
node->slots = 0;
|
|
|
|
|
node->slots_inuse = 0;
|
|
|
|
|
node->slots_alloc = 0;
|
|
|
|
|
node->slots_max = 0;
|
2009-08-11 06:51:27 +04:00
|
|
|
|
|
|
|
|
node->boards = orte_default_num_boards;
|
|
|
|
|
node->sockets_per_board = orte_default_num_sockets_per_board;
|
|
|
|
|
node->cores_per_socket = orte_default_num_cores_per_socket;
|
|
|
|
|
if (NULL != orte_default_cpu_set) {
|
|
|
|
|
node->cpu_set = strdup(orte_default_cpu_set);
|
|
|
|
|
} else {
|
|
|
|
|
node->cpu_set = NULL;
|
|
|
|
|
}
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
node->username = NULL;
|
2009-12-01 02:11:25 +03:00
|
|
|
|
|
|
|
|
OBJ_CONSTRUCT(&node->resources, opal_list_t);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_node_destruct(orte_node_t* node)
|
|
|
|
|
{
|
2009-03-03 19:39:13 +03:00
|
|
|
int i;
|
2009-12-01 02:11:25 +03:00
|
|
|
opal_list_item_t *item;
|
2008-09-01 21:15:01 +04:00
|
|
|
|
|
|
|
|
if (NULL != node->name) {
|
|
|
|
|
free(node->name);
|
2009-07-14 22:56:49 +04:00
|
|
|
node->name = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
2009-01-15 21:11:50 +03:00
|
|
|
if (NULL != node->alias) {
|
|
|
|
|
opal_argv_free(node->alias);
|
2009-07-14 22:56:49 +04:00
|
|
|
node->alias = NULL;
|
2009-01-15 21:11:50 +03:00
|
|
|
}
|
|
|
|
|
|
2009-03-03 19:39:13 +03:00
|
|
|
if (NULL != node->daemon) {
|
|
|
|
|
node->daemon->node = NULL;
|
|
|
|
|
OBJ_RELEASE(node->daemon);
|
2009-07-14 22:56:49 +04:00
|
|
|
node->daemon = NULL;
|
2009-03-03 19:39:13 +03:00
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2009-03-03 19:39:13 +03:00
|
|
|
for (i=0; i < node->procs->size; i++) {
|
|
|
|
|
if (NULL != node->procs->addr[i]) {
|
|
|
|
|
((orte_proc_t*)(node->procs->addr[i]))->node = NULL;
|
|
|
|
|
OBJ_RELEASE(node->procs->addr[i]);
|
2009-07-14 22:56:49 +04:00
|
|
|
node->procs->addr[i] = NULL;
|
2009-03-03 19:39:13 +03:00
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
OBJ_RELEASE(node->procs);
|
|
|
|
|
|
2009-08-11 06:51:27 +04:00
|
|
|
if (NULL != node->cpu_set) {
|
|
|
|
|
free(node->cpu_set);
|
|
|
|
|
node->cpu_set = NULL;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != node->username) {
|
|
|
|
|
free(node->username);
|
2009-07-14 22:56:49 +04:00
|
|
|
node->username = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
2009-12-01 02:11:25 +03:00
|
|
|
|
|
|
|
|
while (NULL != (item = opal_list_remove_first(&node->resources))) {
|
|
|
|
|
OBJ_RELEASE(item);
|
|
|
|
|
}
|
|
|
|
|
OBJ_DESTRUCT(&node->resources);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_node_t,
|
|
|
|
|
opal_list_item_t,
|
|
|
|
|
orte_node_construct,
|
|
|
|
|
orte_node_destruct);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static void orte_proc_construct(orte_proc_t* proc)
|
|
|
|
|
{
|
|
|
|
|
proc->name = *ORTE_NAME_INVALID;
|
|
|
|
|
proc->pid = 0;
|
2009-06-06 05:08:47 +04:00
|
|
|
proc->local_rank = ORTE_LOCAL_RANK_INVALID;
|
|
|
|
|
proc->node_rank = ORTE_NODE_RANK_INVALID;
|
2010-03-24 00:28:02 +03:00
|
|
|
proc->last_errmgr_state = ORTE_PROC_STATE_UNDEF;
|
2008-09-01 21:15:01 +04:00
|
|
|
proc->state = ORTE_PROC_STATE_UNDEF;
|
2010-02-27 20:37:34 +03:00
|
|
|
proc->app_idx = 0;
|
2008-09-01 21:15:01 +04:00
|
|
|
proc->slot_list = NULL;
|
|
|
|
|
proc->node = NULL;
|
|
|
|
|
proc->nodename = NULL;
|
|
|
|
|
proc->rml_uri = NULL;
|
2010-01-07 04:19:44 +03:00
|
|
|
proc->restarts = 0;
|
2010-04-27 02:15:57 +04:00
|
|
|
proc->relocates = 0;
|
2010-03-13 02:57:50 +03:00
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
2008-09-01 21:15:01 +04:00
|
|
|
proc->ckpt_state = 0;
|
|
|
|
|
proc->ckpt_snapshot_ref = NULL;
|
|
|
|
|
proc->ckpt_snapshot_loc = NULL;
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_proc_destruct(orte_proc_t* proc)
|
|
|
|
|
{
|
|
|
|
|
/* do NOT free the nodename field as this is
|
|
|
|
|
* simply a pointer to a field in the
|
|
|
|
|
* associated node object - the node object
|
|
|
|
|
* will free it
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
if (NULL != proc->slot_list) {
|
|
|
|
|
free(proc->slot_list);
|
2009-07-14 22:56:49 +04:00
|
|
|
proc->slot_list = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
2009-07-14 22:56:49 +04:00
|
|
|
if (NULL != proc->node) {
|
|
|
|
|
OBJ_RELEASE(proc->node);
|
|
|
|
|
proc->node = NULL;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2009-07-14 22:56:49 +04:00
|
|
|
if (NULL != proc->rml_uri) {
|
|
|
|
|
free(proc->rml_uri);
|
|
|
|
|
proc->rml_uri = NULL;
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
|
2010-03-13 02:57:50 +03:00
|
|
|
#if OPAL_ENABLE_FT_CR == 1
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != proc->ckpt_snapshot_ref) {
|
|
|
|
|
free(proc->ckpt_snapshot_ref);
|
2009-07-14 22:56:49 +04:00
|
|
|
proc->ckpt_snapshot_ref = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
if (NULL != proc->ckpt_snapshot_loc) {
|
|
|
|
|
free(proc->ckpt_snapshot_loc);
|
2009-07-14 22:56:49 +04:00
|
|
|
proc->ckpt_snapshot_loc = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_proc_t,
|
|
|
|
|
opal_list_item_t,
|
|
|
|
|
orte_proc_construct,
|
|
|
|
|
orte_proc_destruct);
|
|
|
|
|
|
2009-01-07 17:58:38 +03:00
|
|
|
static void orte_attr_construct(orte_attr_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
ptr->name = NULL;
|
|
|
|
|
ptr->size = 0;
|
|
|
|
|
ptr->bytes = NULL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_attr_destruct(orte_attr_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
if (NULL != ptr->name) {
|
|
|
|
|
free(ptr->name);
|
2009-07-14 22:56:49 +04:00
|
|
|
ptr->name = NULL;
|
2009-01-07 17:58:38 +03:00
|
|
|
}
|
|
|
|
|
if (NULL != ptr->bytes) {
|
|
|
|
|
free(ptr->bytes);
|
2009-07-14 22:56:49 +04:00
|
|
|
ptr->bytes = NULL;
|
2009-01-07 17:58:38 +03:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_attr_t,
|
|
|
|
|
opal_list_item_t,
|
|
|
|
|
orte_attr_construct,
|
|
|
|
|
orte_attr_destruct);
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
static void orte_nid_construct(orte_nid_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
ptr->name = NULL;
|
|
|
|
|
ptr->daemon = ORTE_VPID_INVALID;
|
2009-01-07 17:58:38 +03:00
|
|
|
OBJ_CONSTRUCT(&ptr->attrs, opal_list_t);
|
2010-03-23 23:47:41 +03:00
|
|
|
OBJ_CONSTRUCT(&ptr->sysinfo, opal_list_t);
|
2010-05-05 04:48:43 +04:00
|
|
|
#if ORTE_ENABLE_HEARTBEAT
|
|
|
|
|
ptr->beat = 0;
|
|
|
|
|
ptr->missed = 0;
|
|
|
|
|
#endif
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_nid_destruct(orte_nid_t *ptr)
|
|
|
|
|
{
|
2009-01-07 17:58:38 +03:00
|
|
|
opal_list_item_t *item;
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
if (NULL != ptr->name) {
|
|
|
|
|
free(ptr->name);
|
2009-07-14 22:56:49 +04:00
|
|
|
ptr->name = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
2009-01-07 17:58:38 +03:00
|
|
|
while (NULL != (item = opal_list_remove_first(&ptr->attrs))) {
|
|
|
|
|
OBJ_RELEASE(item);
|
|
|
|
|
}
|
|
|
|
|
OBJ_DESTRUCT(&ptr->attrs);
|
2010-03-23 23:47:41 +03:00
|
|
|
while (NULL != (item = opal_list_remove_first(&ptr->sysinfo))) {
|
|
|
|
|
OBJ_RELEASE(item);
|
|
|
|
|
}
|
|
|
|
|
OBJ_DESTRUCT(&ptr->sysinfo);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_nid_t,
|
|
|
|
|
opal_object_t,
|
|
|
|
|
orte_nid_construct,
|
|
|
|
|
orte_nid_destruct);
|
|
|
|
|
|
|
|
|
|
static void orte_pmap_construct(orte_pmap_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
ptr->node = -1;
|
2009-02-06 18:29:33 +03:00
|
|
|
ptr->local_rank = ORTE_LOCAL_RANK_INVALID;
|
|
|
|
|
ptr->node_rank = ORTE_NODE_RANK_INVALID;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_pmap_t,
|
|
|
|
|
opal_object_t,
|
|
|
|
|
orte_pmap_construct,
|
|
|
|
|
NULL);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static void orte_jmap_construct(orte_jmap_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
ptr->job = ORTE_JOBID_INVALID;
|
2009-06-18 08:36:00 +04:00
|
|
|
ptr->num_procs = 0;
|
2009-02-25 05:43:22 +03:00
|
|
|
OBJ_CONSTRUCT(&ptr->pmap, opal_pointer_array_t);
|
|
|
|
|
opal_pointer_array_init(&ptr->pmap,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_MAX_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE);
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_jmap_destruct(orte_jmap_t *ptr)
|
|
|
|
|
{
|
2009-02-25 05:43:22 +03:00
|
|
|
orte_pmap_t **pmaps;
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
pmaps = (orte_pmap_t**)ptr->pmap.addr;
|
|
|
|
|
for (i=0; i < ptr->pmap.size && NULL != pmaps[i]; i++) {
|
|
|
|
|
OBJ_RELEASE(pmaps[i]);
|
|
|
|
|
}
|
2008-09-01 21:15:01 +04:00
|
|
|
OBJ_DESTRUCT(&ptr->pmap);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_jmap_t,
|
|
|
|
|
opal_object_t,
|
|
|
|
|
orte_jmap_construct,
|
|
|
|
|
orte_jmap_destruct);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
static void orte_job_map_construct(orte_job_map_t* map)
|
|
|
|
|
{
|
2009-08-11 06:51:27 +04:00
|
|
|
map->policy = 0;
|
2008-09-01 21:15:01 +04:00
|
|
|
map->npernode = 0;
|
2009-08-11 06:51:27 +04:00
|
|
|
map->nperboard = 0;
|
|
|
|
|
map->npersocket = 0;
|
|
|
|
|
map->cpus_per_rank = 1;
|
|
|
|
|
map->stride = 1;
|
2008-09-01 21:15:01 +04:00
|
|
|
map->oversubscribe = true; /* default to allowing oversubscribe */
|
|
|
|
|
map->display_map = false;
|
|
|
|
|
map->cpu_lists = false;
|
|
|
|
|
map->num_new_daemons = 0;
|
|
|
|
|
map->daemon_vpid_start = ORTE_VPID_INVALID;
|
|
|
|
|
map->num_nodes = 0;
|
|
|
|
|
map->nodes = OBJ_NEW(opal_pointer_array_t);
|
|
|
|
|
opal_pointer_array_init(map->nodes,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_MAX_SIZE,
|
|
|
|
|
ORTE_GLOBAL_ARRAY_BLOCK_SIZE);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void orte_job_map_destruct(orte_job_map_t* map)
|
|
|
|
|
{
|
|
|
|
|
orte_std_cntr_t i;
|
|
|
|
|
|
|
|
|
|
for (i=0; i < map->nodes->size; i++) {
|
|
|
|
|
if (NULL != map->nodes->addr[i]) {
|
|
|
|
|
OBJ_RELEASE(map->nodes->addr[i]);
|
2009-07-14 22:56:49 +04:00
|
|
|
map->nodes->addr[i] = NULL;
|
2008-09-01 21:15:01 +04:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
OBJ_RELEASE(map->nodes);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_job_map_t,
|
|
|
|
|
opal_object_t,
|
|
|
|
|
orte_job_map_construct,
|
|
|
|
|
orte_job_map_destruct);
|
2009-06-17 06:54:20 +04:00
|
|
|
|
|
|
|
|
static void orte_regex_node_construct(orte_regex_node_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
ptr->prefix = NULL;
|
2009-06-18 08:36:00 +04:00
|
|
|
OBJ_CONSTRUCT(&ptr->suffix, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->suffix, sizeof(char));
|
2009-06-17 06:54:20 +04:00
|
|
|
OBJ_CONSTRUCT(&ptr->nodes, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->nodes, sizeof(int32_t));
|
|
|
|
|
OBJ_CONSTRUCT(&ptr->cnt, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->cnt, sizeof(int32_t));
|
|
|
|
|
OBJ_CONSTRUCT(&ptr->starting_vpid, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->starting_vpid, sizeof(orte_vpid_t));
|
|
|
|
|
OBJ_CONSTRUCT(&ptr->ppn, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->ppn, sizeof(int32_t));
|
|
|
|
|
OBJ_CONSTRUCT(&ptr->nrank, opal_value_array_t);
|
|
|
|
|
opal_value_array_init(&ptr->nrank, sizeof(orte_node_rank_t));
|
|
|
|
|
}
|
|
|
|
|
static void orte_regex_node_destruct(orte_regex_node_t *ptr)
|
|
|
|
|
{
|
|
|
|
|
if (NULL != ptr->prefix) {
|
|
|
|
|
free(ptr->prefix);
|
2009-07-14 22:56:49 +04:00
|
|
|
ptr->prefix = NULL;
|
2009-06-17 06:54:20 +04:00
|
|
|
}
|
2009-06-18 08:36:00 +04:00
|
|
|
OBJ_DESTRUCT(&ptr->suffix);
|
2009-06-17 06:54:20 +04:00
|
|
|
OBJ_DESTRUCT(&ptr->nodes);
|
|
|
|
|
OBJ_DESTRUCT(&ptr->cnt);
|
|
|
|
|
OBJ_DESTRUCT(&ptr->starting_vpid);
|
|
|
|
|
OBJ_DESTRUCT(&ptr->ppn);
|
|
|
|
|
OBJ_DESTRUCT(&ptr->nrank);
|
|
|
|
|
}
|
|
|
|
|
OBJ_CLASS_INSTANCE(orte_regex_node_t,
|
|
|
|
|
opal_list_item_t,
|
|
|
|
|
orte_regex_node_construct,
|
|
|
|
|
orte_regex_node_destruct);
|
|
|
|
|
|
2008-09-01 21:15:01 +04:00
|
|
|
#endif
|