1
1
Граф коммитов

138 Коммитов

Автор SHA1 Сообщение Дата
Gleb Natapov
b0614931f4 Remove mpool_tree_item from the mpool_tree before unregistering/freeing memory.
Otherwise a race exists if another thread allocates already freed memory
which is not removed from the mpool_tree yet.

This commit was SVN r16038.
2007-09-03 10:56:55 +00:00
Gleb Natapov
091862a25a Protect access to mca_mpool_base_tree by a lock.
This commit was SVN r16011.
2007-08-30 10:51:02 +00:00
Jeff Squyres
d3f008492f Introduce a new debugging MCA parameter:
mpi_show_mpi_alloc_mem_leaks

When activated, MPI_FINALIZE displays a list of memory allocations
from MPI_ALLOC_MEM that were not freed by MPI_FREE_MEM (in each MPI
process).

 * If set to a positive integer, display only that many leaks.
 * If set to a negative integer, display all leaks.
 * If set to 0, do not show any leaks.

This commit was SVN r15736.
2007-08-01 21:33:25 +00:00
Rainer Keller
830de8ad20 - In ompi/mca/mpool/base/mpool_base_alloc.c, we may miss freeing
value..
   Actually, this string is not used anywhere else, so just have key and
   value on the stack.

This commit was SVN r15675.
2007-07-30 07:30:47 +00:00
Gleb Natapov
88f4018543 Don't fail MPI_Alloc_mem() when no more memory can be registered.
This commit was SVN r15303.
2007-07-08 11:44:58 +00:00
Brian Barrett
80fa8eef6e Don't include malloc.h in mpool/base/base.h because it causes all kinds of problems
when the memory debugging stuff is enabled.  Push it down into the two .c files
that do use it.

This commit was SVN r14779.
2007-05-27 03:55:21 +00:00
Sven Stork
22af6d38e6 - UNexport symbols that shouldn't be needed outside the libraries
- replace #if/#endif with BEGIN/END_C_DECLS
- reformating

This commit was SVN r14669.
2007-05-16 15:46:52 +00:00
Galen Shipman
db63458495 bring disable_sbrk back online, there was a change to properly support AIX
some time ago (last summer) that included checking for M_TRIM_THRESHOLD and
M_MMAP_MAX, unfortunately we didn't include <malloc.h> which is where these
are define, so disabling sbrk for the registration cache has been busted for
some time. 

This commit was SVN r14169.
2007-03-29 16:11:00 +00:00
George Bosilca
78f362d0d6 Be consistent about the definitions of mca_mpool_base_page_size and
mca_mpool_base_page_size_log. They are exported by the mpool/base/base.h,
if some other code need them, then it should include this file
instead of having it's own redefinition of these externals.

This commit was SVN r14156.
2007-03-28 14:14:05 +00:00
Sven Stork
d8a369936e - Fix more symbols that should be exported.
This commit was SVN r13824.
2007-02-27 15:17:17 +00:00
Pavel Shamis
6fe84f581b mpool_base_module_destroy was removing all modules from
a list instead of removing specific one. Fixing the bug.

This commit was SVN r13795.
2007-02-26 16:25:20 +00:00
George Bosilca
79ea6d471b Even less warnings.
This commit was SVN r13429.
2007-02-01 19:27:11 +00:00
Jeff Squyres
52ca6cf86c The mpi_leave_pinned and mpi_leave_pinned_pipeline MCA parameters were
needlessly registered in multiple different places, and none of them
had a good help string.  There was also an inconsistent check for
setting both mpi_leave_pinned and mpi_leave_pinned_pipeline (i.e., it
was only in ob1).  This commit moves the registration of these params
to one central place (ompi/runtime/ompi_mpi_params.c, with all other
mpi_* MCA params) and uses globals to propagate the values as
relevant.  The error check was also moved to the central location to
ensure that we can consistency everywhere.

This commit was SVN r13226.
2007-01-21 14:02:06 +00:00
Gleb Natapov
190e7a27cd Merge with gleb-mpool branch. All RDMA components use same mpool now (rdma).
udapl/openib/vapi/gm mpools a deprecated. rdma mpool has parameter that allows
to limit its size mpool_rdma_rcache_size_limit (default is 0 - unlimited).

This commit was SVN r12878.
2006-12-17 12:26:41 +00:00
George Bosilca
06563b5dec Last set of explicit conversions. We are now close to the zero warnings on
all platforms. The only exceptions (and I will not deal with them
anytime soon) are on Windows:
- the write functions which require the length to be an int when it's
  a size_t on all UNIX variants.
- all iovec manipulation functions where the iov_len is again an int
  when it's a size_t on most of the UNIXes.
As these only happens on Windows, so I think we're set for now :)

This commit was SVN r12215.
2006-10-20 03:57:44 +00:00
Rainer Keller
668902c780 - trivial spelling
This commit was SVN r12139.
2006-10-17 16:34:52 +00:00
George Bosilca
3f0a7cad9e The last patch for Windows support. Mostly casting and conversion to C++ friendly headers.
This commit was SVN r11400.
2006-08-24 16:38:08 +00:00
Brian Barrett
75d32b4013 * It appears that AIX provides a mallopt, but not the options to disable
sbrk and the use of mmap().  So rather than checking just for mallopt(),
  we should also be checking for those defines when determining if we can
  disable giving memory back to the OS or not.

This commit was SVN r11279.
2006-08-21 14:00:30 +00:00
Sven Stork
556d73085d - Introduce the info key "mpool" to specify which mpool to use for the
allocation. This is necessary to detect if the user requests a specific
  mpool for the allocationi. Searching the key values for a specific mpool
  name does not work for the case that the user provides an info object
  without mpool specific information (see Ticket #254).
- In the case that the user provides a info object without requesting a
  specific mpool we use malloc to allocate buffer instead of returning
  NULL (fix for Ticket #254 )

This commit was SVN r11188.
2006-08-15 09:21:40 +00:00
Brian Barrett
5cadbbbf41 Fix for bug #140. If we're leaving things pinned, certain assumptions about
where to look for registrations that were used in the alloc/free code don't
work (because the memory returned from malloc() -- whowever gets around to
calling it) might actually be registered already.  So just call malloc
and free directly and avoid the whole issue when leave pinned is on.  After
all, you have to pay the registration cost sometime, and if leave pinned
is on, you only have to pay it once.  It makes things much simpler to
have that once be at first use rather than during ALLOC_MEM, and as far
as I can read, we're still standards conformant this way.

This commit was SVN r10406.
2006-06-17 18:34:41 +00:00
George Bosilca
aca71521db Complete the move of the mpool registration from opal_list_item_t to the
ompi_free_list_item_t.

This commit was SVN r10354.
2006-06-14 17:43:50 +00:00
Galen Shipman
18dda70fd0 make ompi_free_list_item_t a class..
This will go to the 1.1 branch but will probably require a few changes as
ompi_free_list_t is different in the branch.. 

This commit was SVN r10306.
2006-06-12 16:44:00 +00:00
Brian Barrett
d3257f22d8 * back out Galen's r10300 because it breaks the build. Real fix coming RSN.
This commit was SVN r10303.

The following SVN revision numbers were found above:
  r10300 --> open-mpi/ompi@b0f3745791
2006-06-12 14:38:14 +00:00
Galen Shipman
b0f3745791 declare these as ompi_free_list_item_t's
This needs to go to 1.1

This commit was SVN r10300.
2006-06-12 13:26:15 +00:00
Galen Shipman
9d73217637 These list items are free list items, and should inherit properly..
This commit was SVN r10295.
2006-06-11 20:19:12 +00:00
Galen Shipman
08823e56fa check address before looking for the item in the tree corresponding to the
address.. 
All have been reviewed by brian.. putting in a changeset request.. 

This commit was SVN r10256.
2006-06-08 16:27:59 +00:00
Galen Shipman
636ef0cf6c don't put back null items on the list..
This commit was SVN r10253.
2006-06-08 14:46:41 +00:00
Galen Shipman
429056078a fix numerous late night errors..
1) don't need tree if memory is just malloc'd 
2) fix memory and free list leak.. 
3) deregister first and then free... doh.. 

This commit was SVN r10251.
2006-06-08 14:23:20 +00:00
Galen Shipman
5a2ceda93f a couple of stupid late night mistakes...
This commit was SVN r10250.
2006-06-08 13:39:41 +00:00
Galen Shipman
b42b0bd1af potential fix for ticket #81
Added a tree to track memory allocation from MPI_Alloc_mem, this allows us to
free the registrations in a sane fashion.. also should be faster.. 

This commit was SVN r10248.
2006-06-08 04:29:27 +00:00
Galen Shipman
deb2254c91 1. mpool_free changes to allow null registrations
2. fix for MPI_Free_mem, was calling deregister but never called mpool_free.. so
we leaked memory. Still an open issue here though, if the memory is alloc'd
and the mpool doesn't create and cache a registration, we will never find the
mpool to free with. 

This commit was SVN r9944.
2006-05-16 22:04:31 +00:00
Galen Shipman
5271948ec0 --- opal object changes
add object size to opal class
no longer need the size when allocating a new object as this is stored in
the class structure

--- dr changes 
Previous rev. maintained state on the communicator used for acking duplicate
fragments, but the communicator may be destroyed prior to successfull
delivery of an ack to the peer. We must therefore maintain this state
globally on a per peer, not a per peer, per communicator basis. 
This requires that we use a global rank on the wire and translate this as
appropriate to a local rank within the communicator. 

This commit was SVN r9454.
2006-03-29 16:19:17 +00:00
George Bosilca
dec87e2cea Remove a warning by protecting one of the variables around #if/#endif.
This commit was SVN r9409.
2006-03-24 04:43:53 +00:00
Galen Shipman
adf621fcce enable both mpool_base_use_mem_hooks and mpool_use_mem_hooks, same for
disable_sbrk. 

This commit was SVN r9385.
2006-03-23 16:15:50 +00:00
Galen Shipman
e548f5f8a8 change pml_ob1_leave_pinned_pipeline param to mpi_leave_pinned_pipeline
This commit was SVN r9384.
2006-03-23 15:57:34 +00:00
George Bosilca
4aa343990f Remove the segfault in ompi_info, when we try to destruct a not yet
constructed object.

This commit was SVN r9308.
2006-03-16 16:56:22 +00:00
George Bosilca
612570134f The request management framework has been redesigned. The main idea is
to let the PML (or io, more generally the low level request manager)
to have it's own release function (what was before the req_fini). This
function will only be called from the low level while the req_free will
be called from the upper level (MPI layer) in order to mark the request
as not used by the user anymore.

From the request point of view the requests will be marked as inactive
everytime we read their status (true for persistent as well). As 
MPI_REQUEST_NULL is already marked as inactive, the test and wait functions
are simpler. The drawback is that now we have to change in the
ompi_request_{test|wait} the req_status of the request once we get it's
status.

This commit was SVN r9290.
2006-03-15 22:53:41 +00:00
George Bosilca
4fb373c7e8 HAVE_MALLOPT is defined only when we have it. It's not a 1/0 ttype of define, it's a
defined/undefined one.

This commit was SVN r9221.
2006-03-08 22:29:01 +00:00
George Bosilca
79d25220b6 HAVE_MALLOPT is automatically detected by configure, so the correct
check is not against a value but against the define.

This commit was SVN r9126.
2006-02-23 04:30:24 +00:00
Galen Shipman
4612f70815 conditionally compile mpool_disable_sbrk if mallopt is present.
This commit was SVN r9107.
2006-02-21 22:27:41 +00:00
Galen Shipman
2b40758cf8 remove bug in turning on mem hooks automatically when leave pinned is
specified.

Turn mem hooks on automatically when sbrk is not disabled and
pml_ob1_leave_pinnned_pipeline is specified. 

This commit was SVN r9018.
2006-02-13 22:34:45 +00:00
Brian Barrett
566a050c23 Next step in the project split, mainly source code re-arranging
- move files out of toplevel include/ and etc/, moving it into the
    sub-projects
  - rather than including config headers with <project>/include, 
    have them as <project>
  - require all headers to be included with a project prefix, with
    the exception of the config headers ({opal,orte,ompi}_config.h
    mpi.h, and mpif.h)

This commit was SVN r8985.
2006-02-12 01:33:29 +00:00
Galen Shipman
1c71ab73d1 Move ompi_pointer_array initialization and destruction outside of the memory
callback, note that ompi_pointer_array_remove_all does not call free. 
This needs to go to the release branch 

This commit was SVN r8944.
2006-02-08 22:40:40 +00:00
Galen Shipman
8f8ed186f1 When leave pinned is set, only enable mem hooks if disable sbrk is not set.
This commit was SVN r8908.
2006-02-06 21:22:20 +00:00
Tim Woodall
c0d18c9579 mem hooks must be enabled if leave_pinned is set
This commit was SVN r8906.
2006-02-06 19:44:21 +00:00
Galen Shipman
1e0ea9dd6d Major fixes for the RDMA registration cache (leave_pinned).
This commit fixes issues with HPL runs on node counts > 4. 

This commit was SVN r8793.
2006-01-23 22:51:50 +00:00
George Bosilca
1e48ab533a Remove some left-around from long time ago ...
This commit was SVN r8748.
2006-01-19 07:08:32 +00:00
Brian Barrett
b42bb3d8e8 More memory hooks tweaks... Set a flag on callbacks if the allocation /
deallocation came from the allocator (malloc, fee, etc) or somewhere
else (the user calling mmap/munmap, etc).  Going to be used by Galen
to determine if it is worth searching the allocations tree

Set flag if it is possible to intercept mmap (not always possible
due to a circular dependency between mmap, dlsym, and calloc)

This commit was SVN r8521.
2005-12-16 01:12:45 +00:00
George Bosilca
ce68d752ba Protect the headers
This commit was SVN r8448.
2005-12-11 05:49:29 +00:00
George Bosilca
53235eb34d The Windows protection is called __WINDOWS__ (and it's a mix between WIN32 and _WIN32).
This commit was SVN r8440.
2005-12-10 22:10:39 +00:00
Brian Barrett
79bf8843d2 * update memory hooks interface to allow for callbacks on both allocations
and dealllocations, per request from Galen and Tim

This commit was SVN r8303.
2005-11-29 04:46:14 +00:00
Brian Barrett
8faa1884f0 * The last of the build system optimizations. Combine the component and
component/base Makefile.am files, reducing the time configure spends
  stamping out Makefiles at the end
* Install base_impl.h file when devel-headers are being installed

This commit was SVN r8200.
2005-11-20 01:03:01 +00:00
Brian Barrett
878676218e Rename opal/memory to opal/memoryhooks because XLC++ on Mac OS X is broken.
When compiling C++ code that includes something that looks for the C++
header file "memory" (stupid C++ headers not having .h extensions), it
goes through the header file search path, which includes $(topsrcdir)/opal,
so it finds the directory $(topsrcdir)/opal/memory/ and tries to load
that as the memory header file and all goes downhill.

This commit was SVN r8111.
2005-11-11 00:26:27 +00:00
Jeff Squyres
42ec26e640 Update the copyright notices for IU and UTK.
This commit was SVN r7999.
2005-11-05 19:57:48 +00:00
Brian Barrett
bcebd1b6b7 Fix a couple of places where headers didn't get installed correctly when
--with-devel-headers is given to configure:
  * allocator, rcache, and mpool were putting things in the wrong place
  * timer wasn't installing the inline implementations at all

This commit was SVN r7805.
2005-10-18 20:12:55 +00:00
Tim Woodall
b570c8cad4 need to specify a size, base address will match
This commit was SVN r7798.
2005-10-18 17:01:36 +00:00
Tim Woodall
c05ef28f6e - added routine to ompi_pointer_array to remove array contents
- corrected memory hook callback to catch all allocations (need to optimize this)
- don't attempt to consolidate allocations

This commit was SVN r7600.
2005-10-03 23:29:26 +00:00
Galen Shipman
67d38b7896 Add multi-nic support to openib
Fix connection establishment race in openib 
Other misc 

This commit was SVN r7570.
2005-09-30 22:58:09 +00:00
Galen Shipman
c1f5543f62 need to call mpool_release on all registrations obtained in the pml.
sanity checks 

This commit was SVN r7530.
2005-09-28 04:49:40 +00:00
Galen Shipman
09e67ce4fd fix off by one on up_align_addr, use base and bound instead of base_align and
bound_align.. 

This commit was SVN r7521.
2005-09-27 18:10:44 +00:00
Galen Shipman
af04b3e1ab fix warnings..
This commit was SVN r7515.
2005-09-27 14:23:51 +00:00
Galen Shipman
3c97b3f722 Modified the registration to include a base_align and bound_align for
searching the tree. Modified the memory callback to search the tree at each
page boundary for registrations. This is necessary as an application may
malloc memory and send out of any portion of that memory, even discontiguous
regions. 

This commit was SVN r7510.
2005-09-27 02:01:21 +00:00
Brian Barrett
d9e80d8f2a * increase size of event queue for receives - it was too small to be useful
on a reasonably sized machine
* if no mpool exists, don't try to malloc out an array of 0 bytes

This commit was SVN r7507.
2005-09-25 17:04:03 +00:00
Galen Shipman
9fe5844071 decrement ref count on removal of registration from mru and tree.
add misc asserts to check for proper reference counting. 

ugly hack 1 -- use mallopt to never release memory ala sbrk - this is
commented out in mca_btl_mvapi_component_init

ugly hack 2 -- test registrations comming out of the tree via rcache_find, for
an unknown reason the tree is returning registrations where the address is not
within the base or bound of the registration. If this happens, we return
NULL. 

comment out code to enable mem hooks if leave_pinned is set, note we can do
this via an mca param and will default it to leave_pinned with mem_hooks when
we iron out these issues. 

I am adding a unit test for the rcache. Note that we have a unit test for the
rb tree but the compare function is significantly different than that used for
registrations. After we have tracked down the issues with rcache_rb we will
remove the above hacks. 

This commit was SVN r7499.
2005-09-24 00:24:49 +00:00
Tim Woodall
848f12e7fd if mpi_leave_pinned is enabled - force malloc hooks
This commit was SVN r7491.
2005-09-22 17:27:56 +00:00
Tim Woodall
b404581293 local variables/objects (regs) must be initialized/constructed
This commit was SVN r7485.
2005-09-22 16:18:23 +00:00
Tim Woodall
7acf0a6bdb corrections for MPI_Free_mem
This commit was SVN r7481.
2005-09-22 15:47:33 +00:00
Galen Shipman
b8cb6e1c64 modified mpool module to contain flags - used to determine if the mpool will
be used in MPI_Alloc_mem operations. Note that we found an interesting bug in
which if memory was allocated by the sm mpool (via mmap) and then registered
via the mvapi mpool, the registration would fail on certain systems. 

Added mca param mpool_base_use_mem_hooks, set to 1 to enable the memory hooks
so that memory is deregistered if the user frees it behind our back. This is
only useful if the mca param mpi_leave_pinned is also set to 1. Otherwise all
registrations are deregistered within the MPI library or via
MPI_Free_buf. After testing we should probably set both mpi_leave_pinned and
mpool_base_use_mem_hooks to default to 1. 

This commit was SVN r7415.
2005-09-16 22:22:03 +00:00
Galen Shipman
22dac4983c fix mca_mpool_base_free to use new rcache
This commit was SVN r7398.
2005-09-15 21:09:07 +00:00
Galen Shipman
8b146cec13 Callback shouldn't be static as it is used in more than one .c file.
This commit was SVN r7369.
2005-09-14 13:27:09 +00:00
Galen Shipman
b3db6fc2d1 Add code for mem_free hooks, commented out right now as there are performance
issues. 
In PML-OB1 - fix for sync-sends of zero length. 

This commit was SVN r7362.
2005-09-14 02:17:04 +00:00
Galen Shipman
d932cfd342 merge of rcache work into the trunk.. lotsa fun ;-)..
I regression tested before the merge, I will regression test tonight and
correct issues that might have crept in. 

This commit was SVN r7329.
2005-09-12 22:28:23 +00:00
Brian Barrett
ed56e743b7 * update configure.ac to use the modern version of AC_INIT and
AM_INIT_AUTOMAKE, instead of the deprecated version.
* Work around dumbness in modern AC_INIT that requires the version
  number to be set at autoconf time (instead of at configure time, as
  it was before).  Set the version number, minus the subversion r number,
  at autoconf time.  Override the internal variables to include the r
  number (if needed) at configure time.  Basically, the right thing
  should always happen.  The only place it might not is the version
  reported as part of configure --help will not have an r number.
* Since AM_INIT_AUTOMAKE taks a list of options, no need to specify
  them in all the Makefile.am files.
* Addes support for subdir-objects, meaning that object files are put
  in the directory containing source files, even if the Makefile.am is
  in another directory.  This should start making it feasible to
  reduce the number of Makefile.am files we have in the tree, which
  will greatly reduce the time to run autogen and configure.

This commit was SVN r7211.
2005-09-07 05:54:53 +00:00
Tim Woodall
a3786c4579 dont attempt to deregister if not supported
This commit was SVN r7200.
2005-09-06 18:27:53 +00:00
Jeff Squyres
99c3edab67 Added mca_mpool_base_module_destroy() function so that you can destroy
an mpool module before MPI_FINALIZE (i.e., ompi_mpi_finalize())

This commit was SVN r7165.
2005-09-03 00:07:34 +00:00
Jeff Squyres
3962c53e2e - Add to AM_CPPFLAGS $(OPAL_LTDL_CPPFLAGS) where necessary in order to
add a -I to find the included ltdl.h (vs. a system-installed ltdl.h)
- Clean up kruft in a bunch of Makefile.am's to remove now-unnecessary
  AM_CPPFLAGS settings to get static-components.h for each framework
- Move the component_repository API functions out of opal/mca/base/base.h
  and into opal/mca/base/mca_base_component_repository.h in order to
  decrease unnecessary dependencies (e.g., before this, almost
  everything in the tree depended on ltdl.h, which is unnecessary --
  only a small number of files really need ltdl.h)

This commit was SVN r7127.
2005-09-01 12:16:36 +00:00
Jeff Squyres
cf16a521c8 Ensure to get ompi/include/constants.h
This commit was SVN r6845.
2005-08-12 21:42:07 +00:00
Jeff Squyres
f41e4149fa - Add new mpool base function: lookup by module name. This allows
multiple components to share a single mpool module (e.g., the
  ptl/btl and coll sm components).
- Re-tool the ptl, btl, and coll sm components to first look for the
  target mpool module, and if they don't find it, to create it.
- coll sm component now correctly identifies when it is supposed to
  run or not (i.e., if all the processes in the communicator are on
  the same host).  Now we just need to fill in some algorithms.  :-)

This commit was SVN r6530.
2005-07-15 20:01:35 +00:00
Tim Woodall
088bd37454 added leave_pinned flag to base registration
This commit was SVN r6506.
2005-07-14 22:28:00 +00:00
Tim Woodall
d454cb290c check for null registration
correct count of registered modules

This commit was SVN r6492.
2005-07-14 19:06:46 +00:00
Galen Shipman
dcbda13a72 Various bug fixes..
This commit was SVN r6464.
2005-07-13 21:13:30 +00:00
Brian Barrett
0ae16f2ab7 * add local hook to remove static-components.h in distclean target. The
files are generated by configure, and not part of the tarball, so
  distclean would be the right place to remove them.

This commit was SVN r6390.
2005-07-08 13:54:12 +00:00
Tim Woodall
269a30fb0e typo
This commit was SVN r6364.
2005-07-06 22:38:39 +00:00
Jeff Squyres
6a9c9953bc Remove a bunch of -I's that are no longer necessary with
properly-prefixed static-component.h files.

This commit was SVN r6342.
2005-07-04 18:24:58 +00:00
Brian Barrett
a13166b500 * rename ompi_output to opal_output
This commit was SVN r6329.
2005-07-03 23:31:27 +00:00
Brian Barrett
39dbeeedfb * rename locking code from ompi to opal
This commit was SVN r6327.
2005-07-03 22:45:48 +00:00
Brian Barrett
761402f95f * rename ompi_list to opal_list
This commit was SVN r6322.
2005-07-03 16:22:16 +00:00
Jeff Squyres
959a08bf42 Compromise:
- move mpool and allocator frameworks back to ompi (from opal)
- specialize the ompi_free_list class to use an mpool instance
- un-specialize opal_free_list to *not* use mpool; just use malloc/free

This commit was SVN r6292.
2005-07-02 15:35:34 +00:00