ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность
openmpi/orte/mca/ras/host/ras_host.c

95 строки
2.3 KiB
C
Исходник Обычный вид История

* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
/*
* Copyright (c) 2004-2005 The Trustees of Indiana University.
* All rights reserved.
* Copyright (c) 2004-2005 The Trustees of the University of Tennessee.
* All rights reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
Add UC copyright This commit was SVN r5009.
2005-03-24 12:43:37 +00:00
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "orte_config.h"
#include "include/orte_constants.h"
#include "include/orte_types.h"
#include "mca/ras/base/base.h"
#include "mca/ras/base/ras_base_node.h"
Windows build checkpoint. Many of the changes are include cleanups, and the addition of pls, ras, and rds to libmpi for the tools directory. Some of the functionality is currently stubbed out. Need to return to implement things properly so we actually run under Windows. But I am focusing on just building at the moment :) This commit was SVN r6133.
2005-06-21 22:48:57 +00:00
#include "mca/ras/host/ras_host.h"
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
strcmp is defined on string.h on Linux so we have to include it. This commit was SVN r5898.
2005-05-31 17:41:37 +00:00
#if HAVE_STRING_H
#include <string.h>
#endif /* HAVE_STRING_H */
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
/**
* Discover available (pre-allocated) nodes. Allocate the
* requested number of nodes/process slots to the job.
*
*/
static int orte_ras_host_allocate(orte_jobid_t jobid)
{
* rename ompi_list to opal_list This commit was SVN r6322.
2005-07-03 16:22:16 +00:00
opal_list_t nodes;
opal_list_item_t* item;
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
int rc;
* rename ompi_list to opal_list This commit was SVN r6322.
2005-07-03 16:22:16 +00:00
OBJ_CONSTRUCT(&nodes, opal_list_t);
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
if(ORTE_SUCCESS != (rc = orte_ras_base_node_query(&nodes))) {
goto cleanup;
}
(copied from a mail that has a lengthy description of this commit) I spoke with Tim about this the other day -- he gave me the green light to go ahead with this, but it turned into a bigger job than I thought it would be. I revamped how the default RAS scheduling and round_robin RMAPS mapping occurs. The previous algorithms were pretty brain dead, and ignored the "slots" and "max_slots" tokens in hostfiles. I considered this a big enough problem to fix it for the beta (because there is currently no way to control where processes are launched on SMPs). There's still some more bells and whistles that I'd like to implement, but there's no hurry, and they can go on the trunk at any time. My patches below are for what I considered "essential", and do the following: - honor the "slots" and "max-slots" tokens in the hostfile (and all their synonyms), meaning that we allocate/map until we fill slots, and if there are still more processes to allocate/map, we keep going until we fill max-slots (i.e., only oversubscribe a node if we have to). - offer two different algorithms, currently supported by two new options to orterun. Remember that there are two parts here -- slot allocation and process mapping. Slot allocation controls how many processes we'll be running on a node. After that decision has been made, process mapping effectively controls where the ranks of MPI_COMM_WORLD (MCW) are placed. Some of the examples given below don't make sense unless you remember that there is a difference between the two (which makes total sense, but you have to think about it in terms of both things): 1. "-bynode": allocates/maps one process per node in a round-robin fashion until all slots on the node are taken. If we still have more processes after all slots are taken, then keep going until all max-slots are taken. Examples: - The hostfile: eddie slots=2 max-slots=4 vogon slots=4 max-slots=8 - orterun -bynode -np 6 -hostfile hostfile a.out eddie: MCW ranks 0, 2 vogon: MCW ranks 1, 3, 4, 5 - orterun -bynode -np 8 -hostfile hostfile a.out eddie: MCW ranks 0, 2, 4 vogon: MCW ranks 1, 3, 5, 6, 7 -> the algorithm oversubscribes all nodes "equally" (until each node's max_slots is hit, of course) - orterun -bynode -np 12 -hostfile hostfile a.out eddie: MCW ranks 0, 2, 4, 6 vogon: MCW ranks 1, 3, 5, 7, 8, 9, 10, 11 2. "-byslot" (this is the default if you don't specify -bynode): greedily takes all available slots on a node for a job before moving on to the next node. If we still have processes to allocate/schedule, then oversubscribe all nodes equally (i.e., go round robin on all nodes until each node's max_slots is hit). Examples: - The hostfile eddie slots=2 max-slots=4 vogon slots=4 max-slots=8 - orterun -np 6 -hostfile hostfile a.out eddie: MCW ranks 0, 1 vogon: MCW ranks 2, 3, 4, 5 - orterun -np 8 -hostfile hostfile a.out eddie: MCW ranks 0, 1, 2 vogon: MCW ranks 3, 4, 5, 6, 7 -> the algorithm oversubscribes all nodes "equally" (until max_slots is hit) - orterun -np 12 -hostfile hostfile a.out eddie: MCW ranks 0, 1, 2, 3 vogon: MCW ranks 4, 5, 6, 7, 8, 9, 10, 11 The above examples are fairly contrived, and it's not clear from them that you can get different allocation answers in all cases (the mapping differences are obvious). Consider the following allocation example: - The hostfile eddie count=4 vogon count=4 earth count=4 deep-thought count=4 - orterun -np 8 -hostfile hostfile a.out eddie: 4 slots will be allocated vogon: 4 slots will be allocated earth: no slots allocated deep-thought: no slots allocated - orterun -bynode -np 8 -hostfile hostfile a.out eddie: 2 slots will be allocated vogon: 2 slots will be allocated earth: 2 slots will be allocated deep-thought: 2 slots will be allocated This commit was SVN r5894.
2005-05-31 16:36:53 +00:00
if (0 == strcmp(mca_ras_host_component.schedule_policy, "node")) {
if (ORTE_SUCCESS !=
(rc = orte_ras_base_allocate_nodes_by_node(jobid, &nodes))) {
goto cleanup;
}
} else {
if (ORTE_SUCCESS !=
(rc = orte_ras_base_allocate_nodes_by_slot(jobid, &nodes))) {
goto cleanup;
}
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
}
cleanup:
* rename ompi_list to opal_list This commit was SVN r6322.
2005-07-03 16:22:16 +00:00
while(NULL != (item = opal_list_remove_first(&nodes))) {
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
OBJ_RELEASE(item);
}
OBJ_DESTRUCT(&nodes);
return rc;
}
Quite a range of small changes. ns_replica.c - Removed the error logging since I use this function in orte_init_stage1 to check if we have created a cellid yet or not. ras_types.h & rase_base_node.h - This was an empty file. moved the orte_ras_node_t from base/ras_base_node.h to this file. - Changed the name of orte_ras_base_node_t to orte_ras_node_t to match the naming mechanisms in place. ras.h - Exposed 2 functions: - node_insert: This takes a list of orte_ras_base_node_t's and places them in the Node Segment of the GPR. This is to be used in orte_init_stage1 for singleton processes, and the hostfile parsing (see rds_hostfile.c). This just puts in the appropriate API interface to keep from calling the orte_ras_base_node_insert function directly. - node_query: This is used in hostfile parsing. This just puts in the appropriate API interface to keep from calling the orte_ras_base_node_query function directly. - Touched all of the implemented components to add reference to these new function pointers ras_base_select.c & ras_base_open.c - Add and set the global module reference rds.h - Exposed 1 function: - store_resource: This stores a list of rds_cell_desc_t's to the Resource Segment. This is used in conjunction with the orte_ras.node_insert function in both the orte_init_stage1 for singleton processes and rds_hostfile.c rds_base_select.c & rds_base_open.c - Add and set the global module reference rds_hostfile.c - Added functionality to create a new cellid for each hostfile, placing each entry in the hostfile into the same cellid. Currently this is commented out with the cellid hard coded to 0, with the intention of taking this out once ORTE is able to handle multiple cellid's - Instead of just adding hosts to the Node Segment via a direct call to the ras_base_node_insert() function. First add the hosts to the Resource Segment of the GPR using the orte_rds.store_resource() function then use the API version of orte_ras.node_insert() to store the hosts on the Node Segment. - Add 1 new function pointer to module as required by the API. rds_hostfile_component.c - Converted this to use the new MCA parameter registration orte_init_stage1.c - It is possible that a cellid was not created yet for the current environment. So I put in some logic to test if the cellid 0 existed. If it does then continue, otherwise create the cellid so we can properly interact with the GPR via the RDS. - For the singleton case we insert some 'dummy' data into the GPR. The RAS matches this logic, so I took out the duplicate GPR put logic, and replaced it with a call to the orte_ras.node_insert() function. - Further before calling orte_ras.node_insert() in the singleton case, we also call orte_rds.store_resource() to add the singleton node to the Resource Segment. Console: - Added a bunch of new functions. Still experimenting with many aspects of the implementation. This is a checkpoint, and has very limited functionality. - Should not be considered stable at the moment. This commit was SVN r6813.
2005-08-11 19:51:50 +00:00
static int orte_ras_host_node_insert(opal_list_t *nodes)
{
return orte_ras_base_node_insert(nodes);
}
static int orte_ras_host_node_query(opal_list_t *nodes)
{
return orte_ras_base_node_query(nodes);
}
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
static int orte_ras_host_deallocate(orte_jobid_t jobid)
{
return ORTE_SUCCESS;
}
static int orte_ras_host_finalize(void)
{
return ORTE_SUCCESS;
}
orte_ras_base_module_t orte_ras_host_module = {
orte_ras_host_allocate,
Quite a range of small changes. ns_replica.c - Removed the error logging since I use this function in orte_init_stage1 to check if we have created a cellid yet or not. ras_types.h & rase_base_node.h - This was an empty file. moved the orte_ras_node_t from base/ras_base_node.h to this file. - Changed the name of orte_ras_base_node_t to orte_ras_node_t to match the naming mechanisms in place. ras.h - Exposed 2 functions: - node_insert: This takes a list of orte_ras_base_node_t's and places them in the Node Segment of the GPR. This is to be used in orte_init_stage1 for singleton processes, and the hostfile parsing (see rds_hostfile.c). This just puts in the appropriate API interface to keep from calling the orte_ras_base_node_insert function directly. - node_query: This is used in hostfile parsing. This just puts in the appropriate API interface to keep from calling the orte_ras_base_node_query function directly. - Touched all of the implemented components to add reference to these new function pointers ras_base_select.c & ras_base_open.c - Add and set the global module reference rds.h - Exposed 1 function: - store_resource: This stores a list of rds_cell_desc_t's to the Resource Segment. This is used in conjunction with the orte_ras.node_insert function in both the orte_init_stage1 for singleton processes and rds_hostfile.c rds_base_select.c & rds_base_open.c - Add and set the global module reference rds_hostfile.c - Added functionality to create a new cellid for each hostfile, placing each entry in the hostfile into the same cellid. Currently this is commented out with the cellid hard coded to 0, with the intention of taking this out once ORTE is able to handle multiple cellid's - Instead of just adding hosts to the Node Segment via a direct call to the ras_base_node_insert() function. First add the hosts to the Resource Segment of the GPR using the orte_rds.store_resource() function then use the API version of orte_ras.node_insert() to store the hosts on the Node Segment. - Add 1 new function pointer to module as required by the API. rds_hostfile_component.c - Converted this to use the new MCA parameter registration orte_init_stage1.c - It is possible that a cellid was not created yet for the current environment. So I put in some logic to test if the cellid 0 existed. If it does then continue, otherwise create the cellid so we can properly interact with the GPR via the RDS. - For the singleton case we insert some 'dummy' data into the GPR. The RAS matches this logic, so I took out the duplicate GPR put logic, and replaced it with a call to the orte_ras.node_insert() function. - Further before calling orte_ras.node_insert() in the singleton case, we also call orte_rds.store_resource() to add the singleton node to the Resource Segment. Console: - Added a bunch of new functions. Still experimenting with many aspects of the implementation. This is a checkpoint, and has very limited functionality. - Should not be considered stable at the moment. This commit was SVN r6813.
2005-08-11 19:51:50 +00:00
orte_ras_host_node_insert,
orte_ras_host_node_query,
* results from initial merge of the tim branch into the trunk. Compiles and ompi_info works, but that's all that has been tested. This commit was SVN r4827.
2005-03-14 20:57:21 +00:00
orte_ras_host_deallocate,
orte_ras_host_finalize
};
Копировать постоянную ссылку