and https://svn.open-mpi.org/trac/ompi/ticket/1853, mallopt() hints do
not always work -- it is possible for memory to be returned to the OS
and therefore OMPI's registration cache becomes invalid.
This commit removes all use of mallopt() and uses a different way to
integrate ptmalloc2 than we have done in the past. In particular, we
use almost exactly the same technique as MX:
* Remove all uses of mallopt, to include the opal/memory mallopt
component.
* Name-shift all of OMPI's internal ptmalloc2 public symbols (e.g.,
malloc -> opal_memory_ptmalloc2_malloc).
* At run-time, use the existing glibc allocator malloc hook function
pointers to fully hijack the glibc allocator with our own
name-shifted ptmalloc2.
* Make the decision whether to hijack the glibc allocator ''at run
time'' (vs. at link time, as previous ptmalloc2 integration
attempts have done). Look at the OMPI_MCA_mpi_leave_pinned
and OMPI_MCA_mpi_leave_pinned_pipeline environment variables and
the existence of /sys/class/infiniband to determine if we should
install the hooks or not.
* As an added bonus, we can now tell if libopen-pal is linked
statically or dynamically, and if we're linked statically, we
assume that munmap intercept support doesn't work.
See the opal/mca/memory/ptmalloc2/README-open-mpi.txt file for all the
gory details about the implementation.
Fixes trac:1853.
This commit was SVN r20921.
The following Trac tickets were found above:
Ticket 1853 --> https://svn.open-mpi.org/trac/ompi/ticket/1853
a standalone library named libopenmpi-malloc. Users wanting to
use leave_pinned with ptmalloc2 will now need to link the library
into their application explicitly. All other users will use the
libc-provided allocator instead of Open MPI's ptmalloc2. This change
may be overriden with the configure option enable-ptmalloc2-internal
- The leave_pinned options will now default to using mallopt on
Linux in the cases where ptmalloc2 was not linked in. mallopt
will also only be available if munmap can be intercepted (the
default whenever Open MPI is not compiled with --without-memory-
manager.
- Open MPI will now complain and refuse to use leave_pinned if
no memory intercept / mallopt option is available.
This commit was SVN r18654.
already set. This can annoy compilers that aren't GNUish
* __align is technically a reserved token and IBM XL appears to be doing
something with it that causes compile badness. So use a different
variable name.
refs trac:279
This commit was SVN r11264.
The following Trac tickets were found above:
Ticket 279 --> https://svn.open-mpi.org/trac/ompi/ticket/279
page protection, which causes the pages to be droped, which causes problems
if we don't deregister the pages first. Since memory is cheap in this case
(it is still usable, should ptmalloc2 want it back, and is limited in size),
we just mprotect the pages instead. This solves the dropping pages problem,
and doesn't cause even more calls into the cache code.
Thanks to Gleb Natapov for both finding the problem and giving a fix.
This should go to the v1.0 branch
This commit was SVN r8732.
deallocation came from the allocator (malloc, fee, etc) or somewhere
else (the user calling mmap/munmap, etc). Going to be used by Galen
to determine if it is worth searching the allocations tree
Set flag if it is possible to intercept mmap (not always possible
due to a circular dependency between mmap, dlsym, and calloc)
This commit was SVN r8521.
the ptmalloc2 memory hooks component triggers callbacks for memory
allocation / deallocation. If enabled (the default) it is only when
memory is actually obtained from or released to the OS (so little
malloc calls only trigger callbacks if sbrk is called). If disabled,
callbacks are triggered every time malloc/free/etc. is called
* It turns out that syscall and mmap aren't good friends due to the return
type of mmap and some old legacy issues with syscall functions that
take more than 5 parameters. For now, default to either loading
the symbol from glibc using dlsym or using the __m{un,}map functions.
Thanks to George for finding this.
* Fix some dumb typos in the mmap / munmap catching code
This commit was SVN r8410.
both mmap and munmap), adjusting the configure script so that the
component will only be activated on systems that use ptmalloc2 in the
first place -- ie, Linux
* Remove the malloc_hooks component - it became an unworkable solution
once threads and such were considered.
* Remove malloc_interpose component - it never worked quite right and
was not going to be able to intercept malloc, so it wasn't going to
be useful for OMPI's purposes.
* Update tests a little bit to match recent memory hooks api
issues - still needs a bit of work.
This commit was SVN r8381.
