ports/openmpi
ports/openmpi
1
1
Форкнуть 0
Код Релизы Активность
6e6c370917
openmpi/ompi/mca/coll/tuned/coll_tuned_bcast.c

834 строки
36 KiB
C
Исходник Обычный вид История

snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
/*
Update the copyright notices for IU and UTK. This commit was SVN r7999.
2005-11-05 22:57:48 +03:00
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
* Copyright (c) 2004-2006 The University of Tennessee and The University
Update the copyright notices for IU and UTK. This commit was SVN r7999.
2005-11-05 22:57:48 +03:00
* of Tennessee Research Foundation. All rights
* reserved.
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "ompi_config.h"
#include "mpi.h"
Next step in the project split, mainly source code re-arranging - move files out of toplevel include/ and etc/, moving it into the sub-projects - rather than including config headers with <project>/include, have them as <project> - require all headers to be included with a project prefix, with the exception of the config headers ({opal,orte,ompi}_config.h mpi.h, and mpif.h) This commit was SVN r8985.
2006-02-12 04:33:29 +03:00
#include "ompi/constants.h"
#include "ompi/datatype/datatype.h"
#include "ompi/communicator/communicator.h"
#include "ompi/mca/coll/coll.h"
#include "ompi/mca/coll/base/coll_tags.h"
#include "ompi/mca/pml/pml.h"
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#include "coll_tuned.h"
#include "coll_tuned_topo.h"
Multiple changes -added some alltoall calls (pairwise checked ok, bruck testing) -changes in use of data hung of communicator -making sendrecv call a true inline function -more use ompi_ddt routines This commit was SVN r7337.
2005-09-13 08:28:18 +04:00
#include "coll_tuned_util.h"
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
int
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
ompi_coll_tuned_bcast_intra_generic( void* buffer,
int original_count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
uint32_t count_by_segment,
ompi_coll_tree_t* tree )
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
{
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
int err = 0, line, i;
int rank, size;
int segindex;
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
int num_segments; /* Number of segments */
int sendcount; /* number of elements sent in this segment */
Last set of explicit conversions. We are now close to the zero warnings on all platforms. The only exceptions (and I will not deal with them anytime soon) are on Windows: - the write functions which require the length to be an int when it's a size_t on all UNIX variants. - all iovec manipulation functions where the iov_len is again an int when it's a size_t on most of the UNIXes. As these only happens on Windows, so I think we're set for now :) This commit was SVN r12215.
2006-10-20 07:57:44 +04:00
size_t realsegsize;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
char *tmpbuf;
size_t type_size;
ptrdiff_t extent, lb;
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
ompi_request_t *recv_reqs[2] = {MPI_REQUEST_NULL, MPI_REQUEST_NULL};
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
ompi_request_t **send_reqs = NULL;
#endif
int req_index;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
size = ompi_comm_size(comm);
rank = ompi_comm_rank(comm);
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
assert( size > 1 );
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
ompi_ddt_get_extent (datatype, &lb, &extent);
ompi_ddt_type_size( datatype, &type_size );
num_segments = (original_count + count_by_segment - 1) / count_by_segment;
realsegsize = count_by_segment * extent;
get/handle extent correctly last rank in comm is not always last in tree structure for nonzero roots This commit was SVN r7135.
2005-09-02 02:56:47 +04:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Set the buffer pointers */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
tmpbuf = (char *) buffer;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if( tree->tree_nextsize != 0 ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
send_reqs = (ompi_request_t**)malloc( tree->tree_nextsize *
sizeof(ompi_request_t*) );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#endif
corrected first seg recv for intermediate nodes (or else we lose a request) This commit was SVN r7137.
2005-09-02 04:55:47 +04:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Root code */
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if( rank == root ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/*
For each segment:
- send segment to all children.
The last segment may have less elements than other segments.
*/
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
sendcount = count_by_segment;
for( segindex = 0; segindex < num_segments; segindex++ ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
if( segindex == (num_segments - 1) ) {
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
sendcount = original_count - segindex * count_by_segment;
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
}
for( i = 0; i < tree->tree_nextsize; i++ ) {
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#if defined(COLL_TUNED_BCAST_USE_BLOCKING)
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
err = MCA_PML_CALL(send(tmpbuf, sendcount, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
MCA_PML_BASE_SEND_STANDARD, comm));
#else
err = MCA_PML_CALL(isend(tmpbuf, sendcount, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm,
&send_reqs[i]));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#endif /* COLL_TUNED_BCAST_USE_BLOCKING */
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
/* complete the sends before starting the next sends */
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
err = ompi_request_wait_all( tree->tree_nextsize, send_reqs,
MPI_STATUSES_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#endif /* not COLL_TUNED_BCAST_USE_BLOCKING */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
/* update tmp buffer */
tmpbuf += realsegsize;
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
}
}
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Intermediate nodes code */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
else if( tree->tree_nextsize > 0 ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/*
Create the pipeline.
1) Post the first receive
2) For segments 1 .. num_segments
- post new receive
- wait on the previous receive to complete
- send this data to children
3) Wait on the last segment
4) Compute number of elements in last segment.
5) Send the last segment to children
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
*/
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
req_index = 0;
MCA_PML_CALL(irecv(tmpbuf, count_by_segment, datatype,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
comm, &recv_reqs[req_index]));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
for( segindex = 1; segindex < num_segments; segindex++ ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
req_index = req_index ^ 0x1;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
/* post new irecv */
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
MCA_PML_CALL(irecv( tmpbuf + realsegsize, count_by_segment,
datatype, tree->tree_prev,
MCA_COLL_BASE_TAG_BCAST,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
comm, &recv_reqs[req_index]));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* wait for and forward the previous segment to children */
err = ompi_request_wait( &recv_reqs[req_index ^ 0x1],
MPI_STATUSES_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
for( i = 0; i < tree->tree_nextsize; i++ ) {
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#if defined(COLL_TUNED_BCAST_USE_BLOCKING)
Correctly compute the number of elements in a segment. For broadcast send the correct size for all intermediary nodes. This commit was SVN r12552.
2006-11-11 02:04:50 +03:00
err = MCA_PML_CALL(send(tmpbuf, count_by_segment, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
MCA_PML_BASE_SEND_STANDARD, comm));
#else
Correctly compute the number of elements in a segment. For broadcast send the correct size for all intermediary nodes. This commit was SVN r12552.
2006-11-11 02:04:50 +03:00
err = MCA_PML_CALL(isend(tmpbuf, count_by_segment, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm,
&send_reqs[i]));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#endif /* COLL_TUNED_BCAST_USE_BLOCKING */
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* complete the sends before starting the next iteration */
err = ompi_request_wait_all( tree->tree_nextsize, send_reqs,
MPI_STATUSES_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
#endif /* COLL_TUNED_BCAST_USE_BLOCKING */
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Update the receive buffer */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
tmpbuf += realsegsize;
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
}
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Process the last segment */
err = ompi_request_wait( &recv_reqs[req_index], MPI_STATUSES_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
sendcount = original_count - (num_segments - 1) * count_by_segment;
for( i = 0; i < tree->tree_nextsize; i++ ) {
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#if defined(COLL_TUNED_BCAST_USE_BLOCKING)
err = MCA_PML_CALL(send(tmpbuf, sendcount, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
MCA_PML_BASE_SEND_STANDARD, comm));
#else
err = MCA_PML_CALL(isend(tmpbuf, sendcount, datatype,
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
tree->tree_next[i],
MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm,
&send_reqs[i]));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#endif /* COLL_TUNED_BCAST_USE_BLOCKING */
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
err = ompi_request_wait_all( tree->tree_nextsize, send_reqs,
MPI_STATUSES_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
#endif /* COLL_TUNED_BCAST_USE_BLOCKING */
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
}
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* Leaf nodes */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
else {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/*
Receive all segments from parent in a loop:
1) post irecv for the first segment
2) for segments 1 .. num_segments
- post irecv for the next segment
- wait on the previous segment to arrive
3) wait for the last segment
*/
req_index = 0;
err = MCA_PML_CALL(irecv(tmpbuf, count_by_segment, datatype,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
comm, &recv_reqs[req_index]));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
for( segindex = 1; segindex < num_segments; segindex++ ) {
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
req_index = req_index ^ 0x1;
tmpbuf += realsegsize;
/* post receive for the next segment */
err = MCA_PML_CALL(irecv(tmpbuf, count_by_segment, datatype,
tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
comm, &recv_reqs[req_index]));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
/* wait on the previous segment */
err = ompi_request_wait( &recv_reqs[req_index ^ 0x1],
MPI_STATUS_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
err = ompi_request_wait( &recv_reqs[req_index], MPI_STATUS_IGNORE );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
}
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if( NULL != send_reqs ) free(send_reqs);
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#endif
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
return (MPI_SUCCESS);
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
error_hndl:
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
OPAL_OUTPUT( (ompi_coll_tuned_stream,"%s:%4d\tError occurred %d, rank %2d",
__FILE__, line, err, rank) );
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#if !defined(COLL_TUNED_BCAST_USE_BLOCKING)
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
if( NULL != send_reqs ) free(send_reqs);
Clean up of generic broadcast function: removing unecessary statements and improving comments. This commit was SVN r13955.
2007-03-08 00:59:53 +03:00
#endif
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
return (err);
}
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
int
ompi_coll_tuned_bcast_intra_bintree ( void* buffer,
int count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
uint32_t segsize )
{
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
int segcount = count;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
size_t typelng;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
COLL_TUNED_UPDATE_BINTREE( comm, tuned_module, root );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
/**
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
* Determine number of elements sent per operation.
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
*/
ompi_ddt_type_size( datatype, &typelng );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
COLL_TUNED_COMPUTED_SEGCOUNT( segsize, typelng, segcount );
Remove the warnings about printf data-type mismatch. This commit was SVN r13804.
2007-02-27 01:20:35 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_binary rank %d ss %5d typelng %lu segcount %d",
ompi_comm_rank(comm), segsize, (unsigned long)typelng, segcount));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
return ompi_coll_tuned_bcast_intra_generic( buffer, count, datatype, root, comm, module,
segcount, data->cached_bintree );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
added segmented binary renamed split so we know its a split binary tree (tested, chain/bin/splitbin all pass simple and ibm tests) This commit was SVN r7138.
2005-09-02 06:19:59 +04:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
int
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
ompi_coll_tuned_bcast_intra_pipeline( void* buffer,
int count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
uint32_t segsize )
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
{
Use the macro to compute the number of elements in a segment in both bcast and reduce and update the default values for the variables as required by the comment in the coll_tuned.h file. This commit was SVN r12546.
2006-11-10 23:04:08 +03:00
int segcount = count;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
size_t typelng;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
COLL_TUNED_UPDATE_PIPELINE( comm, tuned_module, root );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
/**
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
* Determine number of elements sent per operation.
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
*/
ompi_ddt_type_size( datatype, &typelng );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
COLL_TUNED_COMPUTED_SEGCOUNT( segsize, typelng, segcount );
Remove the warnings about printf data-type mismatch. This commit was SVN r13804.
2007-02-27 01:20:35 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_pipeline rank %d ss %5d typelng %lu segcount %d",
ompi_comm_rank(comm), segsize, (unsigned long)typelng, segcount));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
return ompi_coll_tuned_bcast_intra_generic( buffer, count, datatype, root, comm, module,
segcount, data->cached_pipeline );
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
}
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
int
ompi_coll_tuned_bcast_intra_chain( void* buffer,
int count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
uint32_t segsize, int32_t chains )
{
Use the macro to compute the number of elements in a segment in both bcast and reduce and update the default values for the variables as required by the comment in the coll_tuned.h file. This commit was SVN r12546.
2006-11-10 23:04:08 +03:00
int segcount = count;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
size_t typelng;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
COLL_TUNED_UPDATE_CHAIN( comm, tuned_module, root, chains );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
/**
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
* Determine number of elements sent per operation.
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
*/
ompi_ddt_type_size( datatype, &typelng );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
COLL_TUNED_COMPUTED_SEGCOUNT( segsize, typelng, segcount );
Remove the warnings about printf data-type mismatch. This commit was SVN r13804.
2007-02-27 01:20:35 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_chain rank %d fo %d ss %5d typelng %lu segcount %d",
ompi_comm_rank(comm), chains, segsize, (unsigned long)typelng, segcount));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
return ompi_coll_tuned_bcast_intra_generic( buffer, count, datatype, root, comm, module,
segcount, data->cached_chain );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
int
ompi_coll_tuned_bcast_intra_binomial( void* buffer,
int count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
uint32_t segsize )
{
Use the macro to compute the number of elements in a segment in both bcast and reduce and update the default values for the variables as required by the comment in the coll_tuned.h file. This commit was SVN r12546.
2006-11-10 23:04:08 +03:00
int segcount = count;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
size_t typelng;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
COLL_TUNED_UPDATE_BMTREE( comm, tuned_module, root );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
/**
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
* Determine number of elements sent per operation.
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
*/
ompi_ddt_type_size( datatype, &typelng );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
COLL_TUNED_COMPUTED_SEGCOUNT( segsize, typelng, segcount );
Remove the warnings about printf data-type mismatch. This commit was SVN r13804.
2007-02-27 01:20:35 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_binomial rank %d ss %5d typelng %lu segcount %d",
ompi_comm_rank(comm), segsize, (unsigned long)typelng, segcount));
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
return ompi_coll_tuned_bcast_intra_generic( buffer, count, datatype, root, comm, module,
segcount, data->cached_bmtree );
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
}
added segmented binary renamed split so we know its a split binary tree (tested, chain/bin/splitbin all pass simple and ibm tests) This commit was SVN r7138.
2005-09-02 06:19:59 +04:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
int
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
ompi_coll_tuned_bcast_intra_split_bintree ( void* buffer,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
int count,
struct ompi_datatype_t* datatype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t* comm,
struct mca_coll_base_module_1_1_0_t *module,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
uint32_t segsize )
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
{
int err=0, line;
int rank, size;
int segindex, i, lr, pair;
int segcount[2]; /* Number of elements sent with each segment */
uint32_t counts[2];
int num_segments[2]; /* Number of segmenets */
int sendcount[2]; /* the same like segcount, except for the last segment */
Last set of explicit conversions. We are now close to the zero warnings on all platforms. The only exceptions (and I will not deal with them anytime soon) are on Windows: - the write functions which require the length to be an int when it's a size_t on all UNIX variants. - all iovec manipulation functions where the iov_len is again an int when it's a size_t on most of the UNIXes. As these only happens on Windows, so I think we're set for now :) This commit was SVN r12215.
2006-10-20 07:57:44 +04:00
size_t realsegsize[2];
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
char *tmpbuf[2];
Look like a big commit but in fact it address only one issue. The way we're working with size and diplacement of data-type. After this patch all data can contain size_t bytes and the displacements are defined as ptrdiff_t. All of the files I was able to compile have been modified to match this requirement. This commit was SVN r12146.
2006-10-18 00:20:58 +04:00
size_t type_size;
ptrdiff_t type_extent, lb;
Various clean ups Changed component so choice of decision functions controlled by mca params (for now fixed decision functions (if statements) default) started fixes for the various bcasts This commit was SVN r7117.
2005-09-01 03:38:09 +04:00
ompi_request_t *base_req, *new_req;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
ompi_coll_tree_t *tree;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
size = ompi_comm_size(comm);
rank = ompi_comm_rank(comm);
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"ompi_coll_tuned_bcast_intra_split_bintree rank %d root %d ss %5d", rank, root, segsize));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (size == 1) {
return MPI_SUCCESS;
}
Big collective commit. I lightly test it, but I think it should be quite stable. Anyway, the default decision functions (for broadcast, reduce and barrier) are based on a high performance network (not TCP). It should give good performance (really good) for any network having the following caracteristics: small latency (5 microseconds) and good bandwidth (more than 1Gb/s). + Cleanup of the reduce algorithms, plus 2 new algorithms (binary and binomial). Now most of the reduce algorithms use a generic tree based function for completing the reduce. + Added macros for computing the trees (they are used for bcast and reduce right now). + Allow the usage of all 5 topologies. + Jelena's implementation of a binary tree that can be used for non commutative operations. Right now only the tree building function is there, it will get activated soon. + Some others minor cleanups. This commit was SVN r12326.
2006-10-27 02:53:05 +04:00
/* setup the binary tree topology. */
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
COLL_TUNED_UPDATE_BINTREE( comm, tuned_module, root );
tree = data->cached_bintree;
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
err = ompi_ddt_type_size( datatype, &type_size );
/* Determine number of segments and number of elements per segment */
counts[0] = count/2;
if (count % 2 != 0) counts[0]++;
counts[1] = count - counts[0];
if ( segsize > 0 ) {
Fix some compiler warnings introduced in r11619. I checked with George: ompi_ddt_type_size() returns a signed int only because of the MPI spec; it will never return a negative value. So casting the return value out of it to a (uint32_t) is safe, and makes the comparisons be between two unsigned values. This commit was SVN r11639. The following SVN revision numbers were found above: r11619 --> open-mpi/ompi@8667648a1b602a2270c09f897e4dba6570545734
2006-09-13 20:42:31 +04:00
/* Note that ompi_ddt_type_size() will never return a negative
value in typelng; it returns an int [vs. an unsigned type]
because of the MPI spec. */
if (segsize < ((uint32_t) type_size)) {
segsize = type_size; /* push segsize up to hold one type */
}
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
segcount[0] = segcount[1] = segsize / type_size;
num_segments[0] = counts[0]/segcount[0];
if ((counts[0] % segcount[0]) != 0) num_segments[0]++;
num_segments[1] = counts[1]/segcount[1];
if ((counts[1] % segcount[1]) != 0) num_segments[1]++;
} else {
segcount[0] = counts[0];
segcount[1] = counts[1];
num_segments[0] = num_segments[1] = 1;
}
/* if the message is too small to be split into segments */
if( (counts[0] == 0 || counts[1] == 0) ||
(segsize > counts[0] * type_size) ||
(segsize > counts[1] * type_size) ) {
/* call linear version here ! */
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
return (ompi_coll_tuned_bcast_intra_chain ( buffer, count, datatype,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
root, comm, module,
segsize, 1 ));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
}
Various clean ups Changed component so choice of decision functions controlled by mca params (for now fixed decision functions (if statements) default) started fixes for the various bcasts This commit was SVN r7117.
2005-09-01 03:38:09 +04:00
err = ompi_ddt_get_extent (datatype, &lb, &type_extent);
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
/* Determine real segment size */
realsegsize[0] = segcount[0] * type_extent;
realsegsize[1] = segcount[1] * type_extent;
/* set the buffer pointers */
tmpbuf[0] = (char *) buffer;
tmpbuf[1] = (char *) buffer+counts[0] * type_extent;
/* Step 1:
Root splits the buffer in 2 and sends segmented message down the branches.
get/handle extent correctly last rank in comm is not always last in tree structure for nonzero roots This commit was SVN r7135.
2005-09-02 02:56:47 +04:00
Left subtree of the tree receives first half of the buffer, while right
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
subtree receives the remaining message.
*/
/* determine if I am left (0) or right (1), (root is right) */
lr = ((rank + size - root)%size + 1)%2;
/* root code */
if( rank == root ) {
/* determine segment count */
sendcount[0] = segcount[0];
sendcount[1] = segcount[1];
/* for each segment */
for (segindex = 0; segindex < num_segments[0]; segindex++) {
/* for each child */
for( i = 0; i < tree->tree_nextsize && i < 2; i++ ) {
if (segindex >= num_segments[i]) { /* no more segments */
continue;
}
/* determine how many elements are being sent in this round */
if(segindex == (num_segments[i] - 1))
sendcount[i] = counts[i] - segindex*segcount[i];
/* send data */
MCA_PML_CALL(send(tmpbuf[i], sendcount[i], datatype,
tree->tree_next[i], MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
/* update tmp buffer */
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
tmpbuf[i] += realsegsize[i];
}
}
}
/* intermediate nodes code */
else if( tree->tree_nextsize > 0 ) {
/* Intermediate nodes:
get/handle extent correctly last rank in comm is not always last in tree structure for nonzero roots This commit was SVN r7135.
2005-09-02 02:56:47 +04:00
* It will receive segments only from one half of the data.
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
* Which one is determined by whether the node belongs to the "left" or "right"
* subtree. Topoloby building function builds binary tree such that
* odd "shifted ranks" ((rank + size - root)%size) are on the left subtree,
* and even on the right subtree.
*
* Create the pipeline. We first post the first receive, then in the loop we
* post the next receive and after that wait for the previous receive to complete
* and we disseminating the data to all children.
*/
sendcount[lr] = segcount[lr];
MCA_PML_CALL(irecv(tmpbuf[lr], sendcount[lr], datatype,
tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
comm, &base_req));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
for( segindex = 1; segindex < num_segments[lr]; segindex++ ) {
/* determine how many elements to expect in this round */
if( segindex == (num_segments[lr] - 1))
sendcount[lr] = counts[lr] - segindex*segcount[lr];
/* post new irecv */
MCA_PML_CALL(irecv( tmpbuf[lr] + realsegsize[lr], sendcount[lr],
datatype, tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
comm, &new_req));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
/* wait for and forward current segment */
err = ompi_request_wait_all( 1, &base_req, MPI_STATUSES_IGNORE );
for( i = 0; i < tree->tree_nextsize; i++ ) { /* send data to children (segcount[lr]) */
MCA_PML_CALL(send( tmpbuf[lr], segcount[lr], datatype,
tree->tree_next[i], MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
} /* end of for each child */
/* upate the base request */
base_req = new_req;
/* go to the next buffer (ie. the one corresponding to the next recv) */
tmpbuf[lr] += realsegsize[lr];
} /* end of for segindex */
/* wait for the last segment and forward current segment */
err = ompi_request_wait_all( 1, &base_req, MPI_STATUSES_IGNORE );
for( i = 0; i < tree->tree_nextsize; i++ ) { /* send data to children */
MCA_PML_CALL(send(tmpbuf[lr], sendcount[lr], datatype,
tree->tree_next[i], MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
} /* end of for each child */
}
/* leaf nodes */
else {
added segmented binary renamed split so we know its a split binary tree (tested, chain/bin/splitbin all pass simple and ibm tests) This commit was SVN r7138.
2005-09-02 06:19:59 +04:00
/* Just consume segments as fast as possible */
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
sendcount[lr] = segcount[lr];
for (segindex = 0; segindex < num_segments[lr]; segindex++) {
/* determine how many elements to expect in this round */
if (segindex == (num_segments[lr] - 1)) sendcount[lr] = counts[lr] - segindex*segcount[lr];
/* receive segments */
MCA_PML_CALL(recv(tmpbuf[lr], sendcount[lr], datatype,
tree->tree_prev, MCA_COLL_BASE_TAG_BCAST,
comm, MPI_STATUS_IGNORE));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
/* update the initial pointer to the buffer */
tmpbuf[lr] += realsegsize[lr];
}
}
/* reset the buffer pointers */
tmpbuf[0] = (char *) buffer;
tmpbuf[1] = (char *) buffer+counts[0] * type_extent;
/* Step 2:
Find your immediate pair (identical node in opposite subtree) and SendRecv
data buffer with them.
The tree building function ensures that
if (we are not root)
if we are in the left subtree (lr == 0) our pair is (rank+1)%size.
if we are in the right subtree (lr == 1) our pair is (rank-1)%size
If we have even number of nodes the rank (size-1) will pair up with root.
*/
if (lr == 0) {
pair = (rank+1)%size;
} else {
pair = (rank+size-1)%size;
}
get/handle extent correctly last rank in comm is not always last in tree structure for nonzero roots This commit was SVN r7135.
2005-09-02 02:56:47 +04:00
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if ( (size%2) != 0 && rank != root) {
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
err = ompi_coll_tuned_sendrecv( tmpbuf[lr], counts[lr], datatype,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
pair, MCA_COLL_BASE_TAG_BCAST,
tmpbuf[(lr+1)%2], counts[(lr+1)%2], datatype,
pair, MCA_COLL_BASE_TAG_BCAST,
comm, MPI_STATUS_IGNORE, rank);
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
} else if ( (size%2) == 0 ) {
/* root sends right buffer to the last node */
if( rank == root ) {
MCA_PML_CALL(send(tmpbuf[1], counts[1], datatype,
(root+size-1)%size, MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD, comm));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
}
/* last node receives right buffer from the root */
get/handle extent correctly last rank in comm is not always last in tree structure for nonzero roots This commit was SVN r7135.
2005-09-02 02:56:47 +04:00
else if (rank == (root+size-1)%size) {
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
MCA_PML_CALL(recv(tmpbuf[1], counts[1], datatype,
root, MCA_COLL_BASE_TAG_BCAST,
comm, MPI_STATUS_IGNORE));
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
}
/* everyone else exchanges buffers */
else {
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
err = ompi_coll_tuned_sendrecv( tmpbuf[lr], counts[lr], datatype,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
pair, MCA_COLL_BASE_TAG_BCAST,
tmpbuf[(lr+1)%2], counts[(lr+1)%2], datatype,
pair, MCA_COLL_BASE_TAG_BCAST,
comm, MPI_STATUS_IGNORE, rank);
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
if (err != MPI_SUCCESS) { line = __LINE__; goto error_hndl; }
}
}
return (MPI_SUCCESS);
error_hndl:
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"%s:%4d\tError occurred %d, rank %2d", __FILE__,line,err,rank));
snapshot as I move machines do not use This commit was SVN r7103.
2005-08-31 05:43:48 +04:00
return (err);
}
added segmented binary renamed split so we know its a split binary tree (tested, chain/bin/splitbin all pass simple and ibm tests) This commit was SVN r7138.
2005-09-02 06:19:59 +04:00
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
/*
* Linear functions are copied from the BASIC coll module
* they do not segment the message and are simple implementations
* but for some small number of nodes and/or small data sizes they
* are just as fast as tuned/tree based segmenting operations
* and as such may be selected by the decision functions
* These are copied into this module due to the way we select modules
* in V1. i.e. in V2 we will handle this differently and so will not
* have to duplicate code.
* GEF Oct05 after asking Jeff.
*/
/* copied function (with appropriate renaming) starts here */
/*
* bcast_lin_intra
*
* Function: - broadcast using O(N) algorithm
* Accepts: - same arguments as MPI_Bcast()
* Returns: - MPI_SUCCESS or error code
*/
int
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
ompi_coll_tuned_bcast_intra_basic_linear (void *buff, int count,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
struct ompi_datatype_t *datatype, int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t *comm,
struct mca_coll_base_module_1_1_0_t *module)
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
{
int i;
int size;
int rank;
int err;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
ompi_request_t **preq;
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
ompi_request_t **reqs = data->mcct_reqs;
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
size = ompi_comm_size(comm);
rank = ompi_comm_rank(comm);
output the right name and stop confusing george This commit was SVN r9215.
2006-03-08 03:40:14 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"ompi_coll_tuned_bcast_intra_basic_linear rank %d root %d", rank, root));
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
/* Non-root receive the data. */
if (rank != root) {
return MCA_PML_CALL(recv(buff, count, datatype, root,
MCA_COLL_BASE_TAG_BCAST, comm,
MPI_STATUS_IGNORE));
}
/* Root sends data to all others. */
for (i = 0, preq = reqs; i < size; ++i) {
if (i == rank) {
continue;
}
err = MCA_PML_CALL(isend_init(buff, count, datatype, i,
MCA_COLL_BASE_TAG_BCAST,
MCA_PML_BASE_SEND_STANDARD,
comm, preq++));
if (MPI_SUCCESS != err) {
return err;
}
}
--i;
/* Start your engines. This will never return an error. */
MCA_PML_CALL(start(i, reqs));
/* Wait for them all. If there's an error, note that we don't
* care what the error was -- just that there *was* an error. The
* PML will finish all requests, even if one or more of them fail.
* i.e., by the end of this call, all the requests are free-able.
* So free them anyway -- even if there was an error, and return
* the error after we free everything. */
err = ompi_request_wait_all(i, reqs, MPI_STATUSES_IGNORE);
/* Free the reqs */
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
ompi_coll_tuned_free_reqs(reqs, i);
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
/* All done */
return err;
}
/* copied function (with appropriate renaming) ends here */
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
/* The following are used by dynamic and forced rules */
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
/* publish details of each algorithm and if its forced/fixed/locked in */
/* as you add methods/algorithms you must update this and the query/map routines */
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
/* this routine is called by the component only */
/* this makes sure that the mca parameters are set to their initial values and perms */
/* module does not call this they call the forced_getvalues routine instead */
added some more linear functions. minor corrections on naming and debug info This commit was SVN r7887.
2005-10-27 03:51:56 +04:00
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
int ompi_coll_tuned_bcast_intra_check_forced_init (coll_tuned_force_algorithm_mca_param_indices_t *mca_param_indices)
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
{
Correctly detect when the specified algorithm is out of range. In this case we reset it to zero. This commit was SVN r12551.
2006-11-11 00:47:07 +03:00
int rc, max_alg = 6, requested_alg;
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
ompi_coll_tuned_forced_max_algorithms[BCAST] = max_alg;
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
rc = mca_base_param_reg_int (&mca_coll_tuned_component.super.collm_version,
"bcast_algorithm_count",
"Number of bcast algorithms available",
false, true, max_alg, NULL);
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
Cleanups. This commit was SVN r12272.
2006-10-24 02:29:17 +04:00
mca_param_indices->algorithm_param_index
= mca_base_param_reg_int(&mca_coll_tuned_component.super.collm_version,
"bcast_algorithm",
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
"Which bcast algorithm is used. Can be locked down to choice of: 0 ignore, 1 basic linear, 2 chain, 3: pipeline, 4: split binary tree, 5: binary tree, 6: binomial tree.",
Cleanups. This commit was SVN r12272.
2006-10-24 02:29:17 +04:00
false, false, 0, NULL);
Correctly detect when the specified algorithm is out of range. In this case we reset it to zero. This commit was SVN r12551.
2006-11-11 00:47:07 +03:00
mca_base_param_lookup_int(mca_param_indices->algorithm_param_index, &(requested_alg));
if( requested_alg > max_alg ) {
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
if( 0 == ompi_comm_rank( MPI_COMM_WORLD ) ) {
opal_output( 0, "Broadcast algorithm #%d is not available (range [0..%d]). Switching back to ignore(0)\n",
Correctly detect when the specified algorithm is out of range. In this case we reset it to zero. This commit was SVN r12551.
2006-11-11 00:47:07 +03:00
requested_alg, max_alg );
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
}
Correctly detect when the specified algorithm is out of range. In this case we reset it to zero. This commit was SVN r12551.
2006-11-11 00:47:07 +03:00
mca_base_param_set_int( mca_param_indices->algorithm_param_index, 0);
Updates & upgrades: - consistent arguments checking (not allowing to select an algorithm which is not available) - consistent way of computing the segcount (number of datatypes by segment). - small cleanups. - more informative debugging messages. This commit was SVN r12545.
2006-11-10 22:54:09 +03:00
}
Cleanups. This commit was SVN r12272.
2006-10-24 02:29:17 +04:00
mca_param_indices->segsize_param_index
= mca_base_param_reg_int(&mca_coll_tuned_component.super.collm_version,
"bcast_algorithm_segmentsize",
"Segment size in bytes used by default for bcast algorithms. Only has meaning if algorithm is forced and supports segmenting. 0 bytes means no segmentation.",
false, false, 0, NULL);
mca_param_indices->tree_fanout_param_index
= mca_base_param_reg_int(&mca_coll_tuned_component.super.collm_version,
"bcast_algorithm_tree_fanout",
"Fanout for n-tree used for bcast algorithms. Only has meaning if algorithm is forced and supports n-tree topo based operation.",
false, false,
ompi_coll_tuned_init_tree_fanout, /* get system wide default */
NULL);
mca_param_indices->chain_fanout_param_index
= mca_base_param_reg_int(&mca_coll_tuned_component.super.collm_version,
"bcast_algorithm_chain_fanout",
"Fanout for chains used for bcast algorithms. Only has meaning if algorithm is forced and supports chain topo based operation.",
false, false,
ompi_coll_tuned_init_chain_fanout, /* get system wide default */
NULL);
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
return (MPI_SUCCESS);
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
}
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
int ompi_coll_tuned_bcast_intra_do_forced(void *buf, int count,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
struct ompi_datatype_t *dtype,
int root,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct ompi_communicator_t *comm,
struct mca_coll_base_module_1_1_0_t *module)
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
{
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
mca_coll_tuned_module_t *tuned_module = (mca_coll_tuned_module_t*) module;
mca_coll_tuned_comm_t *data = tuned_module->tuned_data;
A few small changes that just expanded in the name of neatness... (1) As pointed out by Torsten after Jeff comment that there are 15 collectives yesterday.. nope.. I have 16 but miss counted them in my ifdefs (I had two #11s). Replaces with enum... (2) Added a readonly MCA param for how many backend algorithms are available per collective (used by benchmarker/STS) This allowed me to remove the tuned query internal functions and replace them with ompi_coll_tuned_forced_max_algorithms[COLL]. (3) I was reading the user forced MCA params for the collectives on each comm create (module init) but I then put the values into a global set of variables (like ompi_coll_tuned_reduce_forced_algorithm). To fix this and make the code neater: (a) The component looks up the MCA param indices on Open if dynamic_rules is set via the ompi_coll_tuned_COLLECTIVE_intra_check_forced_init () call. (b) Got rid of the ompi_coll_ompi_coll_tuned_COLLECTIVE_forced_algorithm/segmentsize/etc globals with a struct that is now cached on the module data hung off the communicator. i.e. done right. (c) On module init if dynamic rules enabled we call a general getvalues routine (in coll_tuned_forced.c) to get the CURRENT values using the MCA param indices and then put them on the modules data segment. A shorter version of getvalues exists for barrier which only needs the algorithm choice This commit was SVN r9663.
2006-04-20 03:42:06 +04:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_do_forced algorithm %d",
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
data->user_forced[BCAST].algorithm));
switch (data->user_forced[BCAST].algorithm) {
case (0): return ompi_coll_tuned_bcast_intra_dec_fixed( buf, count, dtype, root, comm, module );
case (1): return ompi_coll_tuned_bcast_intra_basic_linear( buf, count, dtype, root, comm, module );
case (2): return ompi_coll_tuned_bcast_intra_chain( buf, count, dtype, root, comm, module,
data->user_forced[BCAST].segsize,
data->user_forced[BCAST].chain_fanout );
case (3): return ompi_coll_tuned_bcast_intra_pipeline( buf, count, dtype, root, comm, module,
data->user_forced[BCAST].segsize );
case (4): return ompi_coll_tuned_bcast_intra_split_bintree( buf, count, dtype, root, comm, module,
data->user_forced[BCAST].segsize );
case (5): return ompi_coll_tuned_bcast_intra_bintree( buf, count, dtype, root, comm, module,
data->user_forced[BCAST].segsize );
case (6): return ompi_coll_tuned_bcast_intra_binomial( buf, count, dtype, root, comm, module,
data->user_forced[BCAST].segsize );
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
default:
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_do_forced attempt to select algorithm %d when only 0-%d is valid?",
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
data->user_forced[BCAST].algorithm, ompi_coll_tuned_forced_max_algorithms[BCAST]));
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
} /* switch */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
return (MPI_ERR_ARG);
Infastructure changes. started to add static (fixed if) statement based decision rules based on gigE numbers added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis (this is not in the fixed call path but only in the dynamic (at com create) call path). (these params can be used by test suites such as OCC to choice which algorithm they are using). This commit was SVN r7854.
2005-10-25 07:55:58 +04:00
}
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
int ompi_coll_tuned_bcast_intra_do_this(void *buf, int count,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
struct ompi_datatype_t *dtype,
int root,
struct ompi_communicator_t *comm,
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
struct mca_coll_base_module_1_1_0_t *module,
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
int algorithm, int faninout, int segsize)
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
{
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_do_this algorithm %d topo faninout %d segsize %d",
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
algorithm, faninout, segsize));
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
switch (algorithm) {
Update collectives selection logic to allow for multiple components to be used at nce (up to one unique collective module per collective function). Matches r15795:15921 of the tmp/bwb-coll-select branch This commit was SVN r15924. The following SVN revisions from the original message are invalid or inconsistent and therefore were not cross-referenced: r15795 r15921
2007-08-19 07:37:49 +04:00
case (0): return ompi_coll_tuned_bcast_intra_dec_fixed( buf, count, dtype, root, comm, module );
case (1): return ompi_coll_tuned_bcast_intra_basic_linear( buf, count, dtype, root, comm, module );
case (2): return ompi_coll_tuned_bcast_intra_chain( buf, count, dtype, root, comm, module, segsize, faninout );
case (3): return ompi_coll_tuned_bcast_intra_pipeline( buf, count, dtype, root, comm, module, segsize );
case (4): return ompi_coll_tuned_bcast_intra_split_bintree( buf, count, dtype, root, comm, module, segsize );
case (5): return ompi_coll_tuned_bcast_intra_bintree( buf, count, dtype, root, comm, module, segsize );
case (6): return ompi_coll_tuned_bcast_intra_binomial( buf, count, dtype, root, comm, module, segsize );
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
default:
Update to have public symbols be consistent; use new prefix rule (apparently we've been doing this in opal and orte, but not in ompi yet). All public symbols begin with "ompi_coll_tuned_" (not mca_coll_tuned_) except the component struct. Now this component passes the illegal symbol report with no hits. This commit was SVN r8589.
2005-12-22 16:49:33 +03:00
OPAL_OUTPUT((ompi_coll_tuned_stream,"coll:tuned:bcast_intra_do_this attempt to select algorithm %d when only 0-%d is valid?",
Correct the bcast problem where we always did a bcast with segzise of 0. Activate the reduce decision function. Others small updates (mostly TAB to spaces). This commit was SVN r12161.
2006-10-18 06:00:46 +04:00
algorithm, ompi_coll_tuned_forced_max_algorithms[BCAST]));
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
} /* switch */
New architecture for broadcast. A generic broadcast working on a tree description. Most of the bcast algorithms can be completed using this generic function once we create the tree structure. Add all kind of trees. There are 2 versions of the generic bcast function. One using overlapping between receives (for intermediary nodes) and then blocking sends to all childs and another where all sends are non blocking. I still have to figure out which one give the smallest overhead. This commit was SVN r12530.
2006-11-10 08:53:50 +03:00
return (MPI_ERR_ARG);
File based dynamic up and tested... Lots of misc fixes: printfs->opal_output, handles fanin/out correctly for forced ops unused vars, correct calculations on meaning of 'msgsize' for decision functions (varies depending on algorithm), etc This commit was SVN r8113.
2005-11-11 07:49:29 +03:00
}
Копировать постоянную ссылку