- first we setup the connections in the begining with all the peers
- MX does not handle well the case where several peers make connections to the same
destination simultaneously.
So I change the order in which we connect. First we compute our rank in the array,
then in a round-robin fashion we setup connection starting with our left neighboard.
This commit was SVN r8075.
This flag is inherited by all datatypes create with unavailable datatypes. Basically, we let the user create the wrong datatype but we dont let him using it for any pt2pt communications or any pack/unpack.
This commit was SVN r8069.
comm_fill_rest there is no need for calling ompi_set_group_rank, since
we know already the rank of the process in the new comm. In case the
process was not part of the new communicator (rank = MPI_UNDEFINED)
calling this function caused a segfault on some platforms.
This commit was SVN r8060.
Limit the amount of data to be packed to the remaining on the convertor. This make the things a lot simpler in the pack/unpack functions.
This commit was SVN r8028.
Do not ignore the type and extent of the last optimized basic type in some special cases.
Update the last fake END_LOOpP with the correct value for the first_elem_disp field.
This commit was SVN r8023.
never stored on the stack. It is partially stored on the stack depending on the loops
but every time we pack/unpack a basic datatype we take in account again it's displacement.
This approach make the whole logic a lot simpler. In same time I split the big functions
in several basic block.
This commit was SVN r8021.
- if the alignment of wchar is zero then wchar_t is not supported by the OS. We skip it.
- Now that the definition of end_loop change compute the first_elem_description for all
predefined datatypes.
- In debug mode print a list of the datatypes that are not supported by the current
architecture.
This commit was SVN r8020.
We can compute the number of complete datatype that we will advance, update the stack and
then compute the new position taking in acount only the remaining bytes.
This commit was SVN r8019.
one that does not return any value. There are 2 exceptions MPI_Wtick and MPI_Wtime.
For these 2 we can insert the bindings manually.
This commit was SVN r8016.
restoring the PMPI version. A variety of reasons for this:
- mpi.h was blinding using inline in a C header without the configrue mojo
properly set it, as mpi.h doesn't include ompi_config.h. This eventually
would have caused a borked build.
- mpi.h and mpif.h were never updated to not include PMPI_W{tick,time} as
a proper prototype
- The C++ and F90 bindings didn't do the right things when there was no
PMPI version of the C call, but profiling was enabled
- Since we only use gettimeofday, the function call overhead really doesn't
matter
This should probably go to the 1.0 branch
This commit was SVN r8014.
"MPI_OP_foo")
- Remove all the handlers for MPI_REPLACE for general reductions
(it's only defined for MPI_ACCUMULATE, and ACCUMULATE is handled
differently than the other reductions, so it's safe to make all the
maps for REPLACE be empty)
This commit was SVN r8008.
This file is now yet activated. It will became the default after the next
commit. (checkpoint to start testing on other clusters)
This commit was SVN r8006.
MPI_UNSIGNED_LONG_LONG, MPI_LONG_LONG, and MPI_LONG_LONG_INT --
although I already had implementations of all the relevant functions
for these types. Doh!
This commit was SVN r7944.
go through the dynamic decision rule interface.
(forced algorithms are set with MCA params)
fixed some silly verbose output with wrong func name in it etc
updates to fixed dec rules.
This commit was SVN r7940.
modules, if its priority is zero (the default value). Reason for that is
+ if there is no other module with a priority > 0, the hierarchical
collective module has a problem anyway, since it has to rely on the coll
modules of the subcommunicators. On the other hand, if its priority is
zero, it won't be chosen anyway, and we can simply save the
allreduce/allgather and comm_split operations which might occur during
hierarchy detection.
+ to improve the startup times until we have the modex thing which we
discussed with Jeff and Tim in Knoxville in place
- adding an mca parameter indicating a symmetric configuration. This can
speed up startup times, since each process can conclude from its data onto
the data of the other processes -> no need for the allreduce operations. Per
default this parameter is set to "no".
This commit was SVN r7932.
larger than 32K for inter-nodes transfert ... and then they do not support iovecs larger than
16K for inter-node transfert. Therefore we have to set the size of our first fragment to
16K to match both cases.
This commit was SVN r7926.
REDUCE_SCATTER to more thoroughly check the datatype/op combination
to see if it's valid or not. If it's not, print a meaningful error
message rather than "Invalid MPI_Op" indicating what specifically
was wrong (therefore hopefully helping users track down where in the
code the problem is, and/or telling us that there's a reduction
operation combo that we don't support that we should)
- The check for whether a datatype is intrinsic needed to be updated
-- it's not sufficient to check that dtype->id < DT_MAX_PREDEFINED;
you really need to check the PREDEFINED flag on the datatype.
Thanks to George for this fix (only intrinsics have a meaningful
value in dtype->id).
This commit was SVN r7923.
the base send and receive request from the pml_base, we can solve our problem
if we construct the convertor attached to any request in the pml_base_construct
function. At the end of the life time for each request (here life time is
related to one utilisation, without taking in account the cache) we release
all information attached to the convertors in the _FINI macro by calling the
ompi_convertor_cleanup.
This commit was SVN r7910.
to be supported by mellanox vapi.. perhaps this will be supported in the near
future, for now it doesn't hurt to have it in the trunk
Also cleanup the receive descriptor posting macro's..
This commit was SVN r7903.
at the top-level MPI API function. This allows two kinds of
scenarios:
1. MPI_Ireduce(..., op, ...);
MPI_Op_free(op);
MPI_Wait(...);
For the non-blocking collectives that we're someday planning -- to
make them analogous to non-blocking point-to-point stuff.
2. Thread 1:
MPI_Reduce(..., op, ...);
Thread 2:
MPI_Op_free(op);
Granted, for #2 to occur would tread a fine line between a correct and
erroneous MPI program, but it is possible (as long as the Op_free was
*after* MPI_reduce() had started to execute). It's more realistic
with case #1, where the Op_free() could be executed in the same thread
or a different thread.
This commit was SVN r7870.
stage fairly confident that
- it works in most scenarious (with symmetric hierarchies, with asymmetric
hierarchies, wihout hierarchies - it just removes itself)
- it does not create too many problems (I am not aware of any at least)
- it does not slow down startup anymore dramatically (thanks to the fixes of
Brian, Jeff, Tim and a significant reduction in the number of collective
operations in the comm_query)
Any feedback is highly welcome.
This commit was SVN r7868.
quering some of the collective components. Up to now, it just worked
somehow, but now with correct reference counting for ops in place, it
refused :-)
This commit was SVN r7866.
started to add static (fixed if) statement based decision rules based on gigE numbers
added mca params so that a user can force a certain algorithm/segment/topo on a per collective basis
(this is not in the fixed call path but only in the dynamic (at com create) call path).
(these params can be used by test suites such as OCC to choice which algorithm they are using).
This commit was SVN r7854.
Note, we are failing the ring tests in the intel p2p test suite, but we seem
to fail the same tests under the current trunk.. will look into this further.
This commit was SVN r7823.
convertor (when prepared) increase the reference count on the used datatype. This reference count
will be released only when the OBJ_DESTRUCT is called on a convertor. However, having to call
OBJ_CONSTRUCT and OBJ_DESTRUCT on each request every time we want to use it (even when it come
from the cache) is an expensive operation. This can be avoided is the OBJ_DESTRUCT will leave the
convertor in exactly the same state as OBJ_CONSTRUCT. With this approach we just have to call
OBJ_CONSTRUCT for each convertor once when we initially create the request.
This commit was SVN r7813.
memory initialization, call opal_progress() to push any pending events
around and possibly yield the processor if nothing entertaining is happening.
This should probably go to the 1.0 branch.
This commit was SVN r7808.
--with-devel-headers is given to configure:
* allocator, rcache, and mpool were putting things in the wrong place
* timer wasn't installing the inline implementations at all
This commit was SVN r7805.
number of collective operations and simplifies the logic significantly.
- introducing a special case if size of comm == 1, avoiding thus collective
operations as well ( i.e. no need for hierarchies)
- fix for an unsymmetric case. Still to be tested.
This commit was SVN r7799.
originally suggested by Ralf Wildenhues, to try to speed autogen, configure,
and make (and possibly even make install). Use automake's include directive
to drastically reduce the number of Makefile files (although the number of
Makefile.am files is the same - most are just included in a top-level
Makefile.am). Also use an Automake SUBDIRs feature to eliminate the
dynamic-mca tree, which was no longer really needed. This makes adding
a framework easier (since you don't have to remember the dynamic-mca
tree) and makes building faster (as make doesn't have to recurse through
the dynamic-mca tree)
This commit was SVN r7777.
and for all root nodes and passed all tests.
First cut on barrier (which from my perspective does not make sense from the
performance point of view) and on allreduce (which might make sense),
This commit was SVN r7774.
done. This version also doesn't break ompi (at least if its not chosen :-) ).
New features compared to the version from last Thursday (where bcast and
reduce seemed to work in most scenarios):
- clearer internal infrastructure
- ability to handle all root processes with a (hopefully) minimal number of
local leader communicators.
This commit was SVN r7769.
the same (since those are both mandated by MPI and <31 characters),
but change some of the back-end subroutine names so that they are <31
characters and therefore obey the F90 standard. Remove an outdated /
useless (and confusing) script.
This commit was SVN r7764.
(actually a work around for an optimisation in the reduce for not saving ops on the first recv of each segment)
Minor change in topo.
This commit was SVN r7758.
- update the hierarch stuff to use btl's instead of ptl's
- start the new logic regarding how to handle local leader communicators
This commit was SVN r7691.
reduce_inorder() function -- we don't use the tree at all.
- Add more relevant "volatile"'s for the control buffers in the
fragment mpool (and associated casts where necessary)
This commit was SVN r7616.
that on some OSes the iovec struct is not POSIX complian, the iov_len is not a size_t but simply an int.
This patch, add a local variable (type size_t) to use with the memory allocation function, and then put
back the value in the iov_len field.
This commit was SVN r7615.
on others as an char*. Thus the right side of all assignment should be explicitly casted to an void* in
order to avoid any casting complaints from the compilers.
This commit was SVN r7607.
- corrected memory hook callback to catch all allocations (need to optimize this)
- don't attempt to consolidate allocations
This commit was SVN r7600.
- Move the "process 0" logic out of the main loop in reduce to make
the code a bit less complex (at the price of slight code
duplication, but it iss now significantly easier to read)
- Fix problem with uniquenes guarantee in the bootstrap mpool -- using
the CID alone was not sufficient enough to guarantee uniquenes; now
use (CID, rank 0 process name) tuple to check for uniqueness
- Made a few debugging help changes in coll_sm.h; especially helps
debugging on uniprocessors
This commit was SVN r7599.
- Move one base global to the basic component and make it an MCA
parameter
- Convert the basic component to use the new MCA param API
This commit was SVN r7598.
discussed and cleared with Edgar.
Ensure that only processes who will be in the new communicator call
the coll selection function. It is pointless (and Bad in some cases)
for processes who are not in the new communicator to try to select a
coll module for the new communicator.
This commit was SVN r7573.
On thor, if IB is down, we get NULL back from ibv_get_devices(), which then
caused segfaults in dlist_start().
* Pretty-print error message if no HCAs found
This commit was SVN r7557.
any NICs to use
* Make mvapi, gm, and mx components all publish information, even if there
are no NICs available so that modex_recv doesn't hang. If there are no
NICs available, don't set the reachable bit, but don't do anything
to fail. This unfortunately doesn't cover the hangs that will result if
different procs load different sets of components, but it's a start
This commit was SVN r7550.
lower the default priority to 0 so that it's not active unless you
specifically ask for it (this component needs more testing by people
other than me before we unleash it on the public).
This commit was SVN r7545.
Try to detect which form this version of Open IB uses, defaulting to the 5
version if we can't figure it out (the new version has 5 params)
* Only add -lcm if it exists on the system - some versions of Open IB
apparently don't need it.
This commit was SVN r7542.
bits to expose the low bits being set. We were casting from a size_t to a void*
which is not good when working with big endian machines.
This fix makes MPI 2 dynamics work on PPC 64 (tested with a Linux OS).
This commit was SVN r7538.
The following SVN revision numbers were found above:
r7537 --> open-mpi/ompi@fd45714c03
- If we have an NS error, don't return an error -- this function's
purpose is to abort :-)
- s/abort()/exit(1)/ so that we don't drop massive corefiles
This commit was SVN r7524.
a random string of characters as part of the version number (the really
soon to happen 1.0lanl release and the 1.1sc2005 release that we've
talked about). So rather than having alpha and beta fields that must
be numeric values, have a general field that can be any alphanumeric
value.
This commit was SVN r7511.
searching the tree. Modified the memory callback to search the tree at each
page boundary for registrations. This is necessary as an application may
malloc memory and send out of any portion of that memory, even discontiguous
regions.
This commit was SVN r7510.
add misc asserts to check for proper reference counting.
ugly hack 1 -- use mallopt to never release memory ala sbrk - this is
commented out in mca_btl_mvapi_component_init
ugly hack 2 -- test registrations comming out of the tree via rcache_find, for
an unknown reason the tree is returning registrations where the address is not
within the base or bound of the registration. If this happens, we return
NULL.
comment out code to enable mem hooks if leave_pinned is set, note we can do
this via an mca param and will default it to leave_pinned with mem_hooks when
we iron out these issues.
I am adding a unit test for the rcache. Note that we have a unit test for the
rb tree but the compare function is significantly different than that used for
registrations. After we have tracked down the issues with rcache_rb we will
remove the above hacks.
This commit was SVN r7499.
Red Storm is complaining about invalid memory pointer (need to go back
to Linux and look at this with valgrind)
* Turn off send in place for now, so I can run the tests on RS and see if
everything else is ok
This commit was SVN r7497.
* fix dumb bug in btl_portals_get where I using the dest descriptor key instead
of the source descriptor key for the match bits, resulting in a PtlGet() with
the wrong match bits
This commit was SVN r7496.
need to remove old ones from the front
- call deregister to actually remove items from the cache/mru
list and deregister the memory (if not being used)
This commit was SVN r7476.
be used in MPI_Alloc_mem operations. Note that we found an interesting bug in
which if memory was allocated by the sm mpool (via mmap) and then registered
via the mvapi mpool, the registration would fail on certain systems.
Added mca param mpool_base_use_mem_hooks, set to 1 to enable the memory hooks
so that memory is deregistered if the user frees it behind our back. This is
only useful if the mca param mpi_leave_pinned is also set to 1. Otherwise all
registrations are deregistered within the MPI library or via
MPI_Free_buf. After testing we should probably set both mpi_leave_pinned and
mpool_base_use_mem_hooks to default to 1.
This commit was SVN r7415.
Makefile.options
- Sample in each of the three projects of how to link againt the
relevant libraries so that when components are loaded into a parent
process' space, we don't rely on the libopal/liborte/libmpi symbols
being in the parent's public symbol namespace -- instead,
dynamically link to the relevant libraries, allowing the dynamic
linker to pull those libraries in at run-time, if needed
This commit was SVN r7397.
