Clean up the remainder of the size_t references in the runtime itself. Convert to orte_std_cntr_t wherever it makes sense (only avoid those places where the actual memory size is referenced).
Remove the obsolete oob barrier function (we actually obsoleted it a long time ago - just never bothered to clean it up).
I have done my best to go through all the components and catch everything, even if I couldn't test compile them since I wasn't on that type of system. Still, I cannot guarantee that problems won't show up when you test this on specific systems. Usually, these will just show as "warning: comparison between signed and unsigned" notes which are easily fixed (just change a size_t to orte_std_cntr_t).
In some places, people didn't use size_t, but instead used some other variant (e.g., I found several places with uint32_t). I tried to catch all of them, but...
Once we get all the instances caught and fixed, this should once and for all resolve many of the heterogeneity problems.
This commit was SVN r11204.
allocation. This is necessary to detect if the user requests a specific
mpool for the allocationi. Searching the key values for a specific mpool
name does not work for the case that the user provides an info object
without mpool specific information (see Ticket #254).
- In the case that the user provides a info object without requesting a
specific mpool we use malloc to allocate buffer instead of returning
NULL (fix for Ticket #254 )
This commit was SVN r11188.
addition to my design and testing, it was conceptually approved by
Gil, Gleb, Pasha, Brad, and Galen. Functionally [probably somewhat
lightly] tested by Galen. We may still have to shake out some bugs
during the next few months, but it seems to be working for all the
cases that I can throw at it.
Here's a summary of the changes from that branch:
* Move MCA parameter registration to a new file (btl_openib_mca.c):
* Properly check the retun status of registering MCA params
* Check for valid values of MCA parameters
* Make help strings better
* Otherwise, the only default value of an MCA param that was
changed was max_btls; it went from 4 to -1 (meaning: use all
available)
* Properly prototyped internal functions in _component.c
* Made a bunch of functions static that didn't need to be public
* Renamed to remove "mca_" prefix from static functions
* Call new MCA param registration function
* Call new INI file read/lookup/finalize functions
* Updated a bunch of macros to be "BTL_" instead of "ORTE_"
* Be a little more consistent with return values
* Handle -1 for the max_btls MCA param
* Fixed a free() that should have been an OBJ_RELEASE()
* Some re-indenting
* Added INI-file parsing
* New flex file: btl_openib_ini.l
* New default HCA params .ini file (probably to be expanded over
time by other HCA vendors)
* Added more show_help messages for parsing problems
* Read in INI files and cache the values for later lookup
* When component opens an HCA, lookup to see if any corresponding
values were found in the INI files (ID'ed by the HCA vendor_id
and vendor_part_id)
* Added btl_openib_verbose MCA param that shows what the INI-file
stuff does (e.g., shows which MTU your HCA ends up using)
* Added btl_openib_hca_param_files as a colon-delimited list of INI
files to check for values during startup (in order,
left-to-right, just like the MCA base directory param).
* MTU is currently the only value supported in this framework.
* It is not a fatal error if we don't find params for the HCA in
the INI file(s). Instead, just print a warning. New MCA param
btl_openib_warn_no_hca_params_found can be used to disable
printing the warning.
* Add MTU to peer negotiation when making a connection
* Exchange maximum MTU; select the lesser of the two
This commit was SVN r11182.
1. comm_spawn processes by default will inherit the "--prefix" from their parent job. Thus, the "--prefix" provided on the command line will be propagated automatically to any children.
2. application programs can override the default by providing their own "ompi_prefix" in the MPI_Info parameter passed to comm_spawn
This commit was SVN r11143.
r10877:
add warm up connection option.. of course this only warms up the first
eager btl but this should be adequate for now..
r10881:
Consulted with Galen and did a few things:
- Fix the algorithm to actually make the connections that we want
- Rename the MCA param to mpi_preconnect_all
- Cleanup the code a bit:
- move the logic to a separate .c file
- check return codes properly
This commit was SVN r11114.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r10877
r10877
r10881
r10881
full argument checking (allowing that MPI_PROC_NULL is legal, of course).
Only after the argument checking do we shortcut. Fixes trac:237, which
was caused by moving the MPI_PROC_NULL test in MPI_Bsend_init,
but not allowing for MPI_PROC_NULL when checking rank.
This commit was SVN r11108.
The following SVN revision numbers were found above:
r10972 --> open-mpi/ompi@31c66d92aa
The following Trac tickets were found above:
Ticket 237 --> https://svn.open-mpi.org/trac/ompi/ticket/237
on almost all platforms (except OS X... sigh...). This is the merge
of r10846 - 10894 from the tmp/f90-shared branch to the trunk.
This commit was SVN r11103.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r10846
implemented entirely on top of the PML. This allows us to have a
one-sided interface even when we are using the CM PML and MTLs for
point-to-point transport (and therefore not using the BML/BTLs)
* Old pt2pt component was renamed "rdma", as it will soon be having
real RDMA support added to it.
Work was done in a temporary branch. Commit is the result of the
merge command:
svn merge -r10862:11099 https://svn.open-mpi.org/svn/ompi/tmp/bwb-osc-pt2pt
This commit was SVN r11100.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r10862
r11099
did pre-libevent update. The problem is that the behavior of
OPAL_EVLOOP_ONCE was changed by the OMPI team, which them broke things
during the update, so it had to be reverted to the old meaning of
loop until one event occurs. OPAL_EVLOOP_ONELOOP will go through the
event loop once (like EVLOOP_NONBLOCK) but will pause in the event
library for a bit (like EVLOOP_ONCE).
fixes trac:234
This commit was SVN r11081.
The following Trac tickets were found above:
Ticket 234 --> https://svn.open-mpi.org/trac/ompi/ticket/234
users mailing list:
http://www.open-mpi.org/community/lists/users/2006/07/1680.php
Warning: this log message is not for the weak. Read at your own
risk.
The problem was that we had several variables in Fortran common blocks
of various types, but their C counterparts were all of a type
equivalent to a fortran double complex. This didn't seem to matter
for the compilers that we tested, but we never tested static builds
(which is where this problem seems to occur, at least with the Intel
compiler: the linker compilains that the variable in the common block
in the user's .o file was of one size/alignment but the one in the C
library was a different size/alignment).
So this patch fixes the sizes/types of the Fortran common block
variables and their corresponding C instantiations to be of the same
sizes/types.
But wait, there's more.
We recently introduced a fix for the OSX linker where some C versions
of the fortran common block variables (e.g.,
_ompi_fortran_status_ignore) were not being found when linking
ompi_info (!). Further research shows that the code path for
ompi_info to require ompi_fortran_status_ignore is, unfortunately,
necessary (a quirk of how various components pull in different
portions of the code base -- nothing in ompi_info itself requires
fortran or MPI knowledge, of course).
Hence, the real problem was that there was no code path from ompi_info
to the portion of the code base where the C globals corresponding to
the Fortran common block variables were instantiated. This is because
the OSX linker does not automatically pull in .o files that only
contain unintialized global variables; the OSX linker typically only
pulls in a .o file from a library if it either has a function that is
used or have a global variable that is initialized (that's the short
version; lots of details and corner cases omitted). Hence, we changed
the global C variables corresponding to the fortran common blocks to
be initialized, thereby causing the OSX linker to pull them in
automatically -- problem solved. At the same time, we moved the
constants to another .c file with a function, just for good measure.
However, this didn't really solve the problem:
1. The function in the file with the C versions of the fortran common
block variables (ompi/mpi/f77/test_constants_f.c) did not have a
code path that was reachable from ompi_info, so the only reason
that the constants were found (on OSX) was because they were
initialized in the global scope (i.e., causing the OSX compiler to
pull in that .o file).
2. Initializing these variable in the global scope causes problems for
some linkers where -- once all the size/type problems mentioned
above were fixed -- the alignments of fortran common blocks and C
global variables do not match (even though the types of the Fortran
and C variables match -- wow!). Hence, initializing the C
variables would not necessarily match the alignment of what Fortran
expected, and the linker would issue a warning (i.e., the alignment
warnings referenced in the original post).
The solution is two-fold:
1. Move the Fortran variables from test_constants_f.c to
ompi/mpi/runtime/ompi_mpi_init.c where there are other global
constants that *are* initialized (that had nothing to do with
fortran, so the alignment issues described above are not a factor),
and therefore all linkers (including the OSX linker) will pull in
this .o file and find all the symbols that it needs.
2. Do not initialize the C variables corresponding to the Fortran
common blocks in the global scope. Indeed, never initialize them
at all (because we never need their *values* - we only check for
their *locations*). Since nothing is ever written to these
variables (particularly in the global scope), the linker does not
see any alignment differences during initialization, but does make
both the C and Fortran variables have the same addresses (this
method has been working in LAM/MPI for over a decade).
There were some comments here in the OMPI code base and in the LAM
code base that stated/implied that C variables corresponding to
Fortran common blocks had to have the same alignment as the Fortran
common blocks (i.e., 16). There were attempts in both code bases to
ensure that this was true. However, the attempts were wrong (in both
code bases), and I have now read enough Fortran compiler documentation
to convince myself that matching alignments is not required (indeed,
it's beyond our control). As long as C variables corresponding to
Fortran common blocks are not initialized in the global scope, the
linker will "figure it out" and adjust the alignment to whatever is
required (i.e., the greater of the alignments). Specifically (to
counter comments that no longer exist in the OMPI code base but still
exist in the LAM code base):
- there is no need to make attempts to specially align C variables
corresponding to Fortran common blocks
- the types and sizes of C variables corresponding to Fortran common
blocks should match, but do not need to be on any particular
alignment
Finally, as a side effect of this effort, I found a bunch of
inconsistencies with the intent of status/array_of_statuses
parameters. For all the functions that I modified they should be
"out" (not inout).
This commit was SVN r11057.
of send/receives outstanding.
Use ibv_cq_resize if available after initial creation of completion queue if
cq_size is too small (based on number of peers).
This commit was SVN r11053.
make it consistent with the indenting in the rest of the file
(otherwise it was quite difficult to understand -- saw this while I
was reviewing 11039).
This commit was SVN r11042.
object if provided.
The associated value is a comma-separated list of hosts -- which must be
in the initial allocation -- and is used to populate the application
context map.
This commit was SVN r11039.
libevent-1.1a.
svn merge -r10917:11006 https://svn.open-mpi.org/svn/ompi/tmp/libevent-update
This commit was SVN r11022.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r10917
r11006
Reviewed by: Jeff Squyres
Fix for ticket #220. Missing a few C++ methods.
MPI::Datatype::Create_indexed_block
MPI::Datatype::Create_resize
MPI::Datatype::Get_true_extent
This commit was SVN r11010.
functions MPI_Test, MPI_Testany, MPI_Wait, MPI_Waitany
should not reset the status.MPI_ERROR as passed by user.
- This needed implementing the MPI_Waitsome and MPI_Testsome.
This commit was SVN r10980.
- bsend_init: use *request after error-checking
- Always reset the status->cancelled
- cancel, wait: need to check *request for MPI_REQUEST_NULL, not
NULL...
(actually ompi_request_wait handles MPI_PROC_NULL, so no need
to check&set of status_empty in wait.c)
This commit was SVN r10972.
- ensure to initialize the values that we use for fortran constants
(even tough their *values* don't matter -- only their *addresses* do,
but initializing them or not has implications for the OSX linker)
- move the fortran constants to a file with functions in it, because
the OSX linker sometimes does not import global variables from
object files that do not have functions (I'm not even going to
pretend to get all the subtle details about the OSX linker right
here -- it's just "better" to have global variables in object files
with functions that otherwise get pulled in during linker
resolution).
This commit was SVN r10908.
SPAWN[_MULTIPLE] from a singleton (and displays a pretty help message
explaining that you need to use mpirun). This can be removed when
fixes for ORTE come over that allow SPAWN[_MULTIPLE] from singletons.
This commit was SVN r10898.
shared memory segments
* make sure to properly unlink the collectives sm bootstrap area at
shutdown
* Add missing / in the path for the mpool shared memory segment
* make sure to release the common_mmap structure in the SM btl
after unlinking the file during shutdown
This commit was SVN r10886.
MPI_WAIT, MPI_TEST, MPI_WAITANY, and MPI_TESTANY. It isn't really
clear what the standard wants as the return code for these functions,
and this is what Sun MPI, LAM/MPI, and MPICH2 all do.
Fixes trac:172
This commit was SVN r10872.
The following Trac tickets were found above:
Ticket 172 --> https://svn.open-mpi.org/trac/ompi/ticket/172
final datatype not on the shape of the added datatype. The gaps exist if the
extent of the final datatype is not equal to its size.
This commit was SVN r10867.
than $(LN_S). This causes problems with with Windows and probably
elsewhere (re: #200). So use a slightly different trick to get the
right header selected for the MEMCPY and TIMER components.
* Using the same trick used to solve the AC_CONFIG_LINKS problem,
stop using a separate header file for direct calling in the
PML and MTL. This lets me remove some icky code in ompi_mca.m4
that was more fragile than I really liked.
This commit was SVN r10841.
We now set truncation error if we received more than we delivered for both
the OB1 and DR PMLs (the CM PML doesn't need such a fix, as the condition
is set at the MTL level)
This commit was SVN r10812.
The following Trac tickets were found above:
Ticket 172 --> https://svn.open-mpi.org/trac/ompi/ticket/172
all but buffered and persistent requests. Unfortunately we were note able to
reuse the pml_base_request_t as it was just too heavy for our needs. Lots of
code for 2/10 usec ;-)
This commit was SVN r10810.
Keeping the cache misses as low as possible is always a good approach.
The opal_list_t is widely used, it should be a highly optimized class.
The same functionality can be reached with one one sentinel instead
of 2 currently used.
I don't have anything against the STL version, but so far nothing can
compare with the Knuth algorithm. I replace the current implementation
with a modified version of the Knuth algorithm (the one described in
The Art of Computer Programming). As expected, the latency went down.
This commit was SVN r10776.
There was some old code regarding the convertor which does not have to be there
(the problem was corrected a while ago). In the PML we already know how the progress
function is defined, so call the BML progress instead, which will save one function
call.
The macro MCA_PML_OB1_COMPUTE_SEGMENT_LENGTH is already defined in the pml_ob1.h
so it should not be in the endpoint.h.
Remove a double definition of the mca_pml_ob1_progress function in the pml_ob1.h.
This commit was SVN r10775.
I've introduced a race condition - seeing occasional LOCAL_LENGTH errors on the receive side. I think I'm mixing up eager/max somehow - will look at it more on monday.
This commit was SVN r10690.
convertor doesn't handle it properly
continue peeking until we don't get anything else..
close the endpoint before closing the library..
add a blocking send that uses mx_test ..
This commit was SVN r10684.
is the one provided by the user. For the buffered send the real datatype used
for the communication is always MPI_BYTE and the count can be retrieved from
the req_bytes_packed field. This will decrease the size of the request by
one pointer and one size_t (8 bytes or 16 bytes depending on the architecture).
This commit was SVN r10680.
bsend_request_init, but not both. Otherwise, you don't free
some buffer space and end up leaking buffers and ending in
badness
* since you only call alloc() or init(), but not both, need to
restore reference counting in init()
This commit was SVN r10674.
* Fix for two problems introduced by r10661:
1. ensure to use the key ''after'' it is initialized (sigh).
1. handle the case where we free the attrkey before it is fully
initialized (i.e., some other error causes us to free it). In
this case, don't try to remove the key from the hash map,
because it won't exist.
* More accurate zeroing in the keyval constructor
(ompi_attrkey_item_constructor)
* Widen the scope of the alock such that the attrkey destructor does
not need to acquire it. Instead, assume that the caller already
has it.
* Add a comment about why the keyval may get destroyed as the result
of deleting an attribute (so that I don't have to figure it out
again the next time I read this code :-) )
This commit was SVN r10664.
The following SVN revision numbers were found above:
r10661 --> open-mpi/ompi@fdba2c9df0
thread safety. This is likely to be only the first of multiple steps
for complete thread safety in the MPI attribute code. All tests
[continue to] pass the intel and ibm attribute tests.
Also renamed a variable from "attr" to "attrkey" to reflect that it's
a keyval, not an attribute.
This commit was SVN r10661.
interconnects that provide matching logic in the library.
Currently includes support for MX and some support for
Portals
* Fix overuse of proc_pml pointer on the ompi_proc structuer,
splitting into proc_pml for pml data and proc_bml for
the BML endpoint data
* bug fixes in bsend init code, which wasn't being used by
the OB1 or DR PMLs...
This commit was SVN r10642.
This moves the logic to create the symbolic links for:
- mpirun
- mpiexec
- ompi-ps
- ompi-clean
and their respective man pages to the ompi level from
the orte layer.
This is a bit pedantic, but orte shouldn't be doing the
work of ompi since that is a bit of an abstraction break.
Note: need to autogen.sh to get this. Sorry :(
This commit was SVN r10602.
yes this means it WAS possible for two nodes to choice two different algorithms
(discovered by Doug Gregor and figured out by George)
Also changed some names like size to comsize so we know which sizes we are using where
This should be updated in al versions
This commit was SVN r10601.
order the BTL depending on the real latency for the eager protocol. Starting from now, the
latency one can specify for the devices will be in micro-second, while the bandwidth is in Mbs
(as it was before).
This commit was SVN r10566.
mpi_leave_pinned when multiple OpenIB HCA ports are found.
Specifically, if mpi_leave_pinned == 1 and ultiple HCA ports are
found, the MCA parameter btl_openib_max_btls is set to 1. If the MCA
parameter btl_openib_warn_leave_pinned_multi_port is true, emit a
warning that this happened (having an MCA parameter to control the
warning allows users/sysadmins to turn it off instead of being nagged
for every run).
This commit was SVN r10521.
explicitly enabled at run-time with the mca parameter
io_romio_enable_parallel_optimizations set to something non-zero.
This will enable some magic flags in Panasas if the user didn't
set them (either on or off) and do some slightly better things
with strided collective writes.
This commit was SVN r10516.
standard). This macro allow us to specify the length of the fragment. Now we are
able to know how the message is fragmented between the network devices or inside
the communication protocol.
This commit was SVN r10508.
specified check that the put function is available for the BTL. Same safe check for
the GET function. At the end make sure that at least on communication protocol is
specified, otherwise force the send flag.
This commit was SVN r10507.
by the BTL (btl_max_rdma_size). Now the PUT protocol is pipelined even if there
is just one network between the 2 peers. Unfortunately, this problem is present
the 1.1 (no pipeline for the PUT protocol).
This commit was SVN r10499.
time.
UD is connectionless, and as long as peers are statically assigned to QPs,
there is no reason to set up the adressing information lazily.
Lots of code was axed, as endpoints no longer have state. Removed a
number of other elements in the endpoint struct to make it as lightweight
as possible.
I was able to remove an entire function call/branch in the send path,
which I believe is the main contributor to a 2us drop in NetPIPE latency.
Some whitespace cleanups as well.
Passes IBM test suite, and all but certain intel tests that were failing
before the change, over ob1 PML.
This commit was SVN r10494.
Moved a lot of the module-specific init from the component init to the module init.
Try keeping a pointer to reduce indexing, didn't seem to help - leaving in place
for now.
This commit was SVN r10485.
Playing around with OPAL_LIKELY/UNLIKELY, no real gains yet.
Reworked progress() to process many WC's at a time, as well
as immediately repost groups of receive buffers.
This commit was SVN r10481.
was smaller than the CACHE_LINE_SIZE. Here is the version that works.
In fact this works on 2 steps. First we set the element size to something
multiple of the desired alignment. Then when we allocate memory, we compute
the total size, and we will align each of the elements (we allocate
multiple of them every time) to the CACHE_LINE_SIZE.
This commit was SVN r10479.
bytes). The simplest way to make sure they are aligned is to update
the size of the basic element to a multiple of the desired alignment.
It will use a little bit more memory, but the improvements on the SM BTL
seems quite interesting.
This commit was SVN r10478.
cannot include the PMPI_WTIME|WTICK functions in the external and
double precision statements because some compilers complain about
this. Instead, we need to use the macro that is defined by
configure.ac (MPIF_H_PMPI_W_FUNCS). This unfortunately means that we
need to generate mpif.h (in addition to mpif-config.h) because the
"external" statement is toxic to F90 compilers.
This commit was SVN r10464.
with the other methodology even if there are no choice buffers and no
special constants. But it keeps the Makefile.am simple and the
methodology consistent.
This commit was SVN r10462.
was that declaring the type of MPI_WTICK and MPI_TIME in mpif-common.h
would allow the F90 bindings to call through to the back end f77
function and have the right return type. But upon reflection, that's
silly -- we were just declaring the variables MPI_WTICK and MPI_WTIME
that were of type double precision. Duh.
So add some fixed (non-generated) wrapper F90 functions to call the
back-end *C* MPI_WTICK and MPI_TIME functions (vs. the back end *F77*
functions). We have to call the back-end C functions because there's
a name conflict if we try to call the back-end F77 functions -- for
the same reasons that we can't "implicitly" define MPI_WTIME and
MPI_WTICK in the f90 module, we can't call such an implicitly-defined
function. So we had to add new back-end C functions that are directly
callable from Fortran, the easiest implementation of which was to
provide 4 one-line functions for each (rather than muck around with
weak symbols).
This commit was SVN r10448.
1. ompi/mca/btl/udapl/btl_udapl_proc.c should be including
btl_udapl_endpoint.h for mca_btl_udapl_proc_insert function.
2. btl_udapl_endpoint.c it looks like you are using
&endpoint->endpoint_lock when you should use &ep->endpoint_lock in a
OPAL_THREAD_LOCK call.
3. btl_udapl_frag.h has a couple opal_list_item_t's that should be
ompi_free_list_item_t in the _FRAG_ALLOC_{EAGER,MAX} macros.
This commit was SVN r10442.
call to the _UNPACK macro, in the case where the length of the received data
is zero. This might happens on the PUT protocol.
This commit was SVN r10431.
mpi_leave_pinned when multiple OpenIB HCA ports are found.
Specifically, if mpi_leave_pinned == 1 and ultiple HCA ports are
found, the MCA parameter btl_openib_max_btls is set to 1. If the MCA
parameter btl_openib_warn_leave_pinned_multi_port is true, emit a
warning that this happened (having an MCA parameter to control the
warning allows users/sysadmins to turn it off instead of being nagged
for every run).
This commit was SVN r10424.
send/receive queues if there is something available in the short message
rdma queues, then we have to poll *ALL* the rdma queues before exiting,
or we aren't fair about frag reception and fall into degenerate matching
cases.
This commit was SVN r10410.
where to look for registrations that were used in the alloc/free code don't
work (because the memory returned from malloc() -- whowever gets around to
calling it) might actually be registered already. So just call malloc
and free directly and avoid the whole issue when leave pinned is on. After
all, you have to pay the registration cost sometime, and if leave pinned
is on, you only have to pay it once. It makes things much simpler to
have that once be at first use rather than during ALLOC_MEM, and as far
as I can read, we're still standards conformant this way.
This commit was SVN r10406.
correctly compute the final checksum. This is not a bug in the case where
both the sender and the receiver execute EXACTLY the same checksum
computations but is definitively a problem if not (such as the buffered case).
This commit was SVN r10367.
