1
1
openmpi/orte/mca/sstore/base/base.h
Josh Hursey e12ca48cd9 A number of C/R enhancements per RFC below:
http://www.open-mpi.org/community/lists/devel/2010/07/8240.php

Documentation:
  http://osl.iu.edu/research/ft/

Major Changes: 
-------------- 
 * Added C/R-enabled Debugging support. 
   Enabled with the --enable-crdebug flag. See the following website for more information: 
   http://osl.iu.edu/research/ft/crdebug/ 
 * Added Stable Storage (SStore) framework for checkpoint storage 
   * 'central' component does a direct to central storage save 
   * 'stage' component stages checkpoints to central storage while the application continues execution. 
     * 'stage' supports offline compression of checkpoints before moving (sstore_stage_compress) 
     * 'stage' supports local caching of checkpoints to improve automatic recovery (sstore_stage_caching) 
 * Added Compression (compress) framework to support 
 * Add two new ErrMgr recovery policies 
   * {{{crmig}}} C/R Process Migration 
   * {{{autor}}} C/R Automatic Recovery 
 * Added the {{{ompi-migrate}}} command line tool to support the {{{crmig}}} ErrMgr component 
 * Added CR MPI Ext functions (enable them with {{{--enable-mpi-ext=cr}}} configure option) 
   * {{{OMPI_CR_Checkpoint}}} (Fixes trac:2342) 
   * {{{OMPI_CR_Restart}}} 
   * {{{OMPI_CR_Migrate}}} (may need some more work for mapping rules) 
   * {{{OMPI_CR_INC_register_callback}}} (Fixes trac:2192) 
   * {{{OMPI_CR_Quiesce_start}}} 
   * {{{OMPI_CR_Quiesce_checkpoint}}} 
   * {{{OMPI_CR_Quiesce_end}}} 
   * {{{OMPI_CR_self_register_checkpoint_callback}}} 
   * {{{OMPI_CR_self_register_restart_callback}}} 
   * {{{OMPI_CR_self_register_continue_callback}}} 
 * The ErrMgr predicted_fault() interface has been changed to take an opal_list_t of ErrMgr defined types. This will allow us to better support a wider range of fault prediction services in the future. 
 * Add a progress meter to: 
   * FileM rsh (filem_rsh_process_meter) 
   * SnapC full (snapc_full_progress_meter) 
   * SStore stage (sstore_stage_progress_meter) 
 * Added 2 new command line options to ompi-restart 
   * --showme : Display the full command line that would have been exec'ed. 
   * --mpirun_opts : Command line options to pass directly to mpirun. (Fixes trac:2413) 
 * Deprecated some MCA params: 
   * crs_base_snapshot_dir deprecated, use sstore_stage_local_snapshot_dir 
   * snapc_base_global_snapshot_dir deprecated, use sstore_base_global_snapshot_dir 
   * snapc_base_global_shared deprecated, use sstore_stage_global_is_shared 
   * snapc_base_store_in_place deprecated, replaced with different components of SStore 
   * snapc_base_global_snapshot_ref deprecated, use sstore_base_global_snapshot_ref 
   * snapc_base_establish_global_snapshot_dir deprecated, never well supported 
   * snapc_full_skip_filem deprecated, use sstore_stage_skip_filem 

Minor Changes: 
-------------- 
 * Fixes trac:1924 : {{{ompi-restart}}} now recognizes path prefixed checkpoint handles and does the right thing. 
 * Fixes trac:2097 : {{{ompi-info}}} should now report all available CRS components 
 * Fixes trac:2161 : Manual checkpoint movement. A user can 'mv' a checkpoint directory from the original location to another and still restart from it. 
 * Fixes trac:2208 : Honor various TMPDIR varaibles instead of forcing {{{/tmp}}} 
 * Move {{{ompi_cr_continue_like_restart}}} to {{{orte_cr_continue_like_restart}}} to be more flexible in where this should be set. 
 * opal_crs_base_metadata_write* functions have been moved to SStore to support a wider range of metadata handling functionality. 
 * Cleanup the CRS framework and components to work with the SStore framework. 
 * Cleanup the SnapC framework and components to work with the SStore framework (cleans up these code paths considerably). 
 * Add 'quiesce' hook to CRCP for a future enhancement. 
 * We now require a BLCR version that supports {{{cr_request_file()}}} or {{{cr_request_checkpoint()}}} in order to make the code more maintainable. Note that {{{cr_request_file}}} has been deprecated since 0.7.0, so we prefer to use {{{cr_request_checkpoint()}}}. 
 * Add optional application level INC callbacks (registered through the CR MPI Ext interface). 
 * Increase the {{{opal_cr_thread_sleep_wait}}} parameter to 1000 microseconds to make the C/R thread less aggressive. 
 * {{{opal-restart}}} now looks for cache directories before falling back on stable storage when asked. 
 * {{{opal-restart}}} also support local decompression before restarting 
 * {{{orte-checkpoint}}} now uses the SStore framework to work with the metadata 
 * {{{orte-restart}}} now uses the SStore framework to work with the metadata 
 * Remove the {{{orte-restart}}} preload option. This was removed since the user only needs to select the 'stage' component in order to support this functionality. 
 * Since the '-am' parameter is saved in the metadata, {{{ompi-restart}}} no longer hard codes {{{-am ft-enable-cr}}}. 
 * Fix {{{hnp}}} ErrMgr so that if a previous component in the stack has 'fixed' the problem, then it should be skipped. 
 * Make sure to decrement the number of 'num_local_procs' in the orted when one goes away. 
 * odls now checks the SStore framework to see if it needs to load any checkpoint files before launching (to support 'stage'). This separates the SStore logic from the --preload-[binary|files] options. 
 * Add unique IDs to the named pipes established between the orted and the app in SnapC. This is to better support migration and automatic recovery activities. 
 * Improve the checks for 'already checkpointing' error path. 
 * A a recovery output timer, to show how long it takes to restart a job 
 * Do a better job of cleaning up the old session directory on restart. 
 * Add a local module to the autor and crmig ErrMgr components. These small modules prevent the 'orted' component from attempting a local recovery (Which does not work for MPI apps at the moment) 
 * Add a fix for bounding the checkpointable region between MPI_Init and MPI_Finalize. 

This commit was SVN r23587.

The following Trac tickets were found above:
  Ticket 1924 --> https://svn.open-mpi.org/trac/ompi/ticket/1924
  Ticket 2097 --> https://svn.open-mpi.org/trac/ompi/ticket/2097
  Ticket 2161 --> https://svn.open-mpi.org/trac/ompi/ticket/2161
  Ticket 2192 --> https://svn.open-mpi.org/trac/ompi/ticket/2192
  Ticket 2208 --> https://svn.open-mpi.org/trac/ompi/ticket/2208
  Ticket 2342 --> https://svn.open-mpi.org/trac/ompi/ticket/2342
  Ticket 2413 --> https://svn.open-mpi.org/trac/ompi/ticket/2413
2010-08-10 20:51:11 +00:00

148 строки
5.6 KiB
C

/*
* Copyright (c) 2010 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#ifndef ORTE_SSTORE_BASE_H
#define ORTE_SSTORE_BASE_H
#include "orte_config.h"
#if !ORTE_DISABLE_FULL_SUPPORT
#include "orte/mca/rml/rml.h"
#endif
#include "orte/mca/sstore/sstore.h"
/*
* Global functions for MCA overall SStore
*/
BEGIN_C_DECLS
/**
* Initialize the SSTORE MCA framework
*
* @retval ORTE_SUCCESS Upon success
* @retval ORTE_ERROR Upon failures
*
* This function is invoked during orte_init();
*/
ORTE_DECLSPEC int orte_sstore_base_open(void);
#if !ORTE_DISABLE_FULL_SUPPORT
/**
* Select an available component.
*
* @retval ORTE_SUCCESS Upon Success
* @retval ORTE_NOT_FOUND If no component can be selected
* @retval ORTE_ERROR Upon other failure
*
*/
ORTE_DECLSPEC int orte_sstore_base_select(void);
/**
* Finalize the SSTORE MCA framework
*
* @retval ORTE_SUCCESS Upon success
* @retval ORTE_ERROR Upon failures
*
* This function is invoked during orte_finalize();
*/
ORTE_DECLSPEC int orte_sstore_base_close(void);
/**
* Object stuff
*/
void orte_sstore_base_local_snapshot_info_construct(orte_sstore_base_local_snapshot_info_t *snapshot);
void orte_sstore_base_local_snapshot_info_destruct( orte_sstore_base_local_snapshot_info_t *snapshot);
void orte_sstore_base_global_snapshot_info_construct(orte_sstore_base_global_snapshot_info_t *snapshot);
void orte_sstore_base_global_snapshot_info_destruct( orte_sstore_base_global_snapshot_info_t *snapshot);
/**
* Globals
*/
ORTE_DECLSPEC extern int orte_sstore_base_output;
ORTE_DECLSPEC extern opal_list_t orte_sstore_base_components_available;
ORTE_DECLSPEC extern orte_sstore_base_component_t orte_sstore_base_selected_component;
ORTE_DECLSPEC extern orte_sstore_base_module_t orte_sstore;
/*
* Context of this module
*/
#define ORTE_SSTORE_UNASSIGN_TYPE 0
#define ORTE_SSTORE_GLOBAL_TYPE 1
#define ORTE_SSTORE_LOCAL_TYPE 2
#define ORTE_SSTORE_TOOL_TYPE 4
#define ORTE_SSTORE_APP_TYPE 8
ORTE_DECLSPEC extern int orte_sstore_context;
/**
* Snapshot metadata
*/
#define SSTORE_METADATA_LOCAL_CRS_COMP_STR CRS_METADATA_COMP
#define SSTORE_METADATA_LOCAL_PID_STR CRS_METADATA_PID
#define SSTORE_METADATA_LOCAL_CONTEXT_STR CRS_METADATA_CONTEXT
#define SSTORE_METADATA_LOCAL_MKDIR_STR CRS_METADATA_MKDIR
#define SSTORE_METADATA_LOCAL_TOUCH_STR CRS_METADATA_TOUCH
#define SSTORE_METADATA_LOCAL_COMPRESS_COMP_STR ("# OPAL Compress Component: ")
#define SSTORE_METADATA_LOCAL_COMPRESS_POSTFIX_STR ("# OPAL Compress Postfix: ")
#define SSTORE_METADATA_LOCAL_SNAP_REF_FMT_STR ("# Local Snapshot Format Reference: ")
#define SSTORE_METADATA_GLOBAL_SNAP_SEQ_STR ("# Seq: ")
#define SSTORE_METADATA_GLOBAL_AMCA_PARAM_STR ("# AMCA: ")
#define SSTORE_METADATA_INTERNAL_DONE_SEQ_STR ("# Finished Seq: ")
#define SSTORE_METADATA_INTERNAL_TIME_STR ("# Timestamp: ")
#define SSTORE_METADATA_INTERNAL_PROCESS_STR ("# Process: ")
#define SSTORE_METADATA_INTERNAL_MIG_SEQ_STR ("# Migrate Seq: ")
#define SSTORE_METADATA_INTERNAL_DONE_MIG_SEQ_STR ("# Finished Migrate Seq: ")
/**
* Some utility functions
*/
ORTE_DECLSPEC extern bool orte_sstore_base_is_checkpoint_available;
ORTE_DECLSPEC extern char * orte_sstore_base_local_metadata_filename;
ORTE_DECLSPEC extern char * orte_sstore_base_global_metadata_filename;
ORTE_DECLSPEC extern char * orte_sstore_base_local_snapshot_fmt;
ORTE_DECLSPEC extern char * orte_sstore_base_global_snapshot_dir;
ORTE_DECLSPEC extern char * orte_sstore_base_global_snapshot_ref;
ORTE_DECLSPEC extern char * orte_sstore_base_prelaunch_location;
ORTE_DECLSPEC int orte_sstore_base_get_global_snapshot_ref(char **name_str, pid_t pid);
ORTE_DECLSPEC int orte_sstore_base_convert_key_to_string(orte_sstore_base_key_t key, char **key_str);
ORTE_DECLSPEC int orte_sstore_base_convert_string_to_key(char *key_str, orte_sstore_base_key_t *key);
ORTE_DECLSPEC int orte_sstore_base_metadata_read_next_seq_num(FILE *file);
ORTE_DECLSPEC int orte_sstore_base_metadata_read_next_token(FILE *file, char **token, char **value);
ORTE_DECLSPEC int orte_sstore_base_metadata_seek_to_seq_num(FILE *file, int seq_num);
ORTE_DECLSPEC int orte_sstore_base_extract_global_metadata(orte_sstore_base_global_snapshot_info_t *global_snapshot);
ORTE_DECLSPEC int orte_sstore_base_get_all_snapshots(opal_list_t *all_snapshots, char *basedir);
ORTE_DECLSPEC int orte_sstore_base_find_largest_seq_num(orte_sstore_base_global_snapshot_info_t *global_snapshot, int *seq_num);
ORTE_DECLSPEC int orte_sstore_base_find_all_seq_nums(orte_sstore_base_global_snapshot_info_t *global_snapshot, int *num_seq, char ***seq_list);
/*
* Common Tool functionality for interfacing with orte-restart/checkpoint
*/
ORTE_DECLSPEC int orte_sstore_base_tool_request_restart_handle(orte_sstore_base_handle_t *handle,
char *basedir, char *ref, int seq,
orte_sstore_base_global_snapshot_info_t *snapshot);
ORTE_DECLSPEC int orte_sstore_base_tool_get_attr(orte_sstore_base_handle_t handle, orte_sstore_base_key_t key, char **value);
#endif /* ORTE_DISABLE_FULL_SUPPORT */
END_C_DECLS
#endif /* ORTE_SSTORE_BASE_H */