ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность
openmpi/orte/mca/ras/base/ras_private.h

115 строки
3.0 KiB
C
Исходник Обычный вид История

Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
Replace all occurences of orte_pointer_array by opal_pointer_array. Remove the implementation of orte_pointer_array. This commit was SVN r17636.
2008-02-28 05:32:23 +00:00
* Copyright (c) 2004-2008 The University of Tennessee and The University
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
* Copyright (c) 2008 UT-Battelle, LLC. All rights reserved.
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
/** @file:
*/
#ifndef ORTE_RAS_PRIVATE_H
#define ORTE_RAS_PRIVATE_H
/*
* includes
*/
#include "orte_config.h"
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
#include "orte/constants.h"
#include "orte/types.h"
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
This commit looks a lot bigger than it is, so relax :-) Fix the problem observed by multiple people that comm_spawned children were (once again) being mapped onto the same nodes as their parents. This was caused by going through the RAS a second time, thus overwriting the mapper's bookkeeping that told RMAPS where it had left off. To solve this - and to continue moving forward on the ORTE development - we introduce the concept of attributes to control the behavior of the RM frameworks. I defined the attributes and a list of attributes as new ORTE data types to make it easier for people to pass them around (since they are now fundamental to the system, and therefore we will be packing and unpacking them frequently). Thus, all the functions to manipulate attributes can be implemented and debugged in one place. I used those capabilities in two places: 1. Added an attribute list to the rmgr.spawn interface. 2. Added an attribute list to the ras.allocate interface. At the moment, the only attribute I modified the various RAS components to recognize is the USE_PARENT_ALLOCATION one (as defined in rmgr_types.h). So the RAS components now know how to reuse an allocation. I have debugged this under rsh, but it now needs to be tested on a wider set of platforms. This commit was SVN r12138.
2006-10-17 16:06:17 +00:00
#include "opal/class/opal_list.h"
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
#include "orte/mca/ras/ras_types.h"
#include "orte/mca/ras/ras.h"
#include "orte/mca/ras/base/base.h"
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
BEGIN_C_DECLS
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/*
* API function definitions
*/
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
ORTE_DECLSPEC int orte_ras_base_allocate(orte_job_t *jdata);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
/**
* Add the specified node definitions to the registry
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
*/
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
ORTE_DECLSPEC int orte_ras_base_node_insert(opal_list_t*, orte_job_t*);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
#if 0
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/*
* Internal support functions
*/
ORTE_DECLSPEC int orte_ras_base_allocate_nodes(orte_jobid_t jobid,
opal_list_t* nodes);
This commit looks a lot bigger than it is, so relax :-) Fix the problem observed by multiple people that comm_spawned children were (once again) being mapped onto the same nodes as their parents. This was caused by going through the RAS a second time, thus overwriting the mapper's bookkeeping that told RMAPS where it had left off. To solve this - and to continue moving forward on the ORTE development - we introduce the concept of attributes to control the behavior of the RM frameworks. I defined the attributes and a list of attributes as new ORTE data types to make it easier for people to pass them around (since they are now fundamental to the system, and therefore we will be packing and unpacking them frequently). Thus, all the functions to manipulate attributes can be implemented and debugged in one place. I used those capabilities in two places: 1. Added an attribute list to the rmgr.spawn interface. 2. Added an attribute list to the ras.allocate interface. At the moment, the only attribute I modified the various RAS components to recognize is the USE_PARENT_ALLOCATION one (as defined in rmgr_types.h). So the RAS components now know how to reuse an allocation. I have debugged this under rsh, but it now needs to be tested on a wider set of platforms. This commit was SVN r12138.
2006-10-17 16:06:17 +00:00
ORTE_DECLSPEC int orte_ras_base_reallocate(orte_jobid_t parent_jobid,
orte_jobid_t child_jobid);
Restore the paffinity capability, along with all the required logic to ensure we "do the right thing" when the user gives us inaccurate information about the number of slots on a remote node. This commit was SVN r12780.
2006-12-06 15:59:34 +00:00
ORTE_DECLSPEC int orte_ras_base_set_oversubscribe_override(orte_jobid_t job);
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
ORTE_DECLSPEC int orte_ras_base_hostfile_query(char *hostfile);
Squeeeeeeze the launch message. This is the message sent to the daemons that provides all the data required for launching their local procs. In reorganizing the ODLS framework, I discovered that we were sending a significant amount of unnecessary and repeated data. This commit resolves this by: 1. taking advantage of the fact that we no longer create the launch message via a GPR trigger. In earlier times, we had the GPR create the launch message based on a subscription. In that mode of operation, we could not guarantee the order in which the data was stored in the message - hence, we had no choice but to parse the message in a loop that checked each value against a list of possible "keys" until the corresponding value was found. Now, however, we construct the message "by hand", so we know precisely what data is in each location in the message. Thus, we no longer need to send the character string "keys" for each data value any more. This represents a rather large savings in the message size - to give you an example, we typically would use a 30-char "key" for a 2-byte data value. As you can see, the overhead can become very large. 2. sending node-specific data only once. Again, because we used to construct the message via subscriptions that were done on a per-proc basis, the data for each node (e.g., the daemon's name, whether or not the node was oversubscribed) would be included in the data for each proc. Thus, the node-specific data was repeated for every proc. Now that we construct the message "by hand", there is no reason to do this any more. Instead, we can insert the data for a specific node only once, and then provide the per-proc data for that node. We therefore not only save all that extra data in the message, but we also only need to parse the per-node data once. The savings become significant at scale. Here is a comparison between the revised trunk and the trunk prior to this commit (all data was taken on odin, using openib, 64 nodes, unity message routing, tested with application consisting of mpi_init/mpi_barrier/mpi_finalize, all execution times given in seconds, all launch message sizes in bytes): Per-node scaling, taken at 1ppn: #nodes original trunk revised trunk time size time size 1 0.10 819 0.09 564 2 0.14 1070 0.14 677 3 0.15 1321 0.14 790 4 0.15 1572 0.15 903 8 0.17 2576 0.20 1355 16 0.25 4584 0.21 2259 32 0.28 8600 0.27 4067 64 0.50 16632 0.39 7683 Per-proc scaling, taken at 64 nodes ppn original trunk revised trunk time size time size 1 0.50 16669 0.40 7720 2 0.55 32733 0.54 11048 3 0.87 48797 0.81 14376 4 1.0 64861 0.85 17704 Condensing those numbers, it appears we gained: per-node message size: 251 bytes/node -> 113 bytes/node per-proc message size: 251 bytes/proc -> 52 bytes/proc per-job message size: 568 bytes/job -> 399 bytes/job (job-specific data such as jobid, override oversubscribe flag, total #procs in job, total slots allocated) The fact that the two pre-commit trunk numbers are the same confirms the fact that each proc was containing the node data as well. It isn't quite the 10x message reduction I had hoped to get, but it is significant and gives much better scaling. Note that the timing info was, as usual, pretty chaotic - the numbers cited here were typical across several runs taken after the initial one to avoid NFS file positioning influences. Also note that this commit removes the orte_process_info.vpid_start field and the handful of places that passed that useless value. By definition, all jobs start at vpid=0, so all we were doing is passing "0" around. In fact, many places simply hardwired it to "0" anyway rather than deal with it. This commit was SVN r16428.
2007-10-11 15:57:26 +00:00
ORTE_DECLSPEC int orte_ras_base_get_oversubscribe_override(orte_jobid_t job, bool *flag);
Add ORTE ALPS support (Cray XT CNL) This commit was SVN r17482.
2008-02-17 19:29:06 +00:00
ORTE_DECLSPEC int orte_ras_base_read_nodename_file(opal_list_t *nodes, char *filename);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/*
* Query the registry for all available nodes
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_query(opal_list_t*);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/*
* Query the registry for a specific node
*/
reapply r15517 and r15520, which were removed in r15527 so that I could get the RML/OOB merge in slightly easier This commit was SVN r15530. The following SVN revision numbers were found above: r15517 --> open-mpi/ompi@41977fcc95e1a1b5658e3ca7a43fa4fadb801dcc r15520 --> open-mpi/ompi@9cbc9df1b8c1f95df52d1c8e7ba4cf1c2605e4bc r15527 --> open-mpi/ompi@2d17dd95169a96febbf38eef48912bf38fc65aa9
2007-07-20 02:34:29 +00:00
ORTE_DECLSPEC orte_ras_node_t* orte_ras_base_node_lookup(const char* nodename);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/**
* Query the registry for all nodes allocated to a specific job
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_query_alloc(opal_list_t*, orte_jobid_t);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
Roll in the Voltaire core/socket/etc process mapping implementation. Only change I made was to cleanup some of the diagnostic output in the odls_default component so it uses the -mca odls_base_verbose parameter. You will not see any impact from this change unless you use the syntax described in ticket #1023. I've tried as many of the RAS components as possible and saw no problem - there may be issues with other RAS components that would not compile on any of my systems. Anything that appears should be trivial to fix. This commit was SVN r15427.
2007-07-14 15:14:07 +00:00
ORTE_DECLSPEC int orte_ras_base_proc_query_alloc(opal_list_t* procs);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/**
* Add the specified node definitions to the registry
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_insert(opal_list_t*);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
reapply r15517 and r15520, which were removed in r15527 so that I could get the RML/OOB merge in slightly easier This commit was SVN r15530. The following SVN revision numbers were found above: r15517 --> open-mpi/ompi@41977fcc95e1a1b5658e3ca7a43fa4fadb801dcc r15520 --> open-mpi/ompi@9cbc9df1b8c1f95df52d1c8e7ba4cf1c2605e4bc r15527 --> open-mpi/ompi@2d17dd95169a96febbf38eef48912bf38fc65aa9
2007-07-20 02:34:29 +00:00
ORTE_DECLSPEC int orte_ras_base_proc_insert(opal_list_t* procs, orte_jobid_t jobid);
Roll in the Voltaire core/socket/etc process mapping implementation. Only change I made was to cleanup some of the diagnostic output in the odls_default component so it uses the -mca odls_base_verbose parameter. You will not see any impact from this change unless you use the syntax described in ticket #1023. I've tried as many of the RAS components as possible and saw no problem - there may be issues with other RAS components that would not compile on any of my systems. Anything that appears should be trivial to fix. This commit was SVN r15427.
2007-07-14 15:14:07 +00:00
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/**
* Delete the specified nodes from the registry
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_delete(opal_list_t*);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/**
* Assign the allocated slots on the specified nodes to the
* indicated jobid.
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_assign(opal_list_t*, orte_jobid_t);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
/**
* Check to see if the node segment is empty
*/
ORTE_DECLSPEC what needs to be ORTE_DECLSPES. This commit was SVN r11997.
2006-10-05 05:22:22 +00:00
ORTE_DECLSPEC int orte_ras_base_node_segment_empty(bool *empty);
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
#endif
Merge the ORTE devel branch into the main trunk. Details of what this means will be circulated separately. Remains to be tested to ensure everything came over cleanly, so please continue to withhold commits a little longer This commit was SVN r17632.
2008-02-28 01:57:57 +00:00
END_C_DECLS
Here is the major MAD-cure commit. I have written plenty about it, so I refer you here to those messages for a description of everything that was done. This commit was SVN r11661.
2006-09-14 21:29:51 +00:00
#endif
Копировать постоянную ссылку