cleanup code in the signal part of the event library
* Only attempt to forward standard input if we have a controlling terminal
(isatty() returns 1) and we are the foreground process OR we do not have
a controlling terminal (isatty() returns 0). If we have a controlling
terminal, check at each SIGCONT if we should change our forwarding,
since our foreground / background status may have changed.
Unfortunately, there isn't a great way in the iof framework to know if
we are capturing a starter's stdin. Use the logic that if it's a source
AND tagged as standard input, it's a starter's stdin. This seems to
work for all the common usages.
Both these need to go to the v1.0 branch.
This commit was SVN r8894.
makes illegal free() calls behave in a much more rational way. You'll still
probably die, but your stack trace will not have 3 billion pages of recusion
inside the memory allocator.
* Fix illegal free in the opal_wrapper code. basename() returns a string in a
static buffer, so it shouldn't be free()ed. It also shouldn't be left around
so long, as another call to basename() may whack the returned buffer. So
leave the free and add a strdup() around the basename() call.
* Turn off some unneeded debugging in the opal_wrapper code that would list the
comamnd to be run, regardless of the -showme option.
This commit was SVN r8758.
A small and ugly workaround the path problem on windows (the absolute
path start with [a-z]: whn : is considered as separator for most of
the environment variables.
This commit was SVN r8746.
page protection, which causes the pages to be droped, which causes problems
if we don't deregister the pages first. Since memory is cheap in this case
(it is still usable, should ptmalloc2 want it back, and is limited in size),
we just mprotect the pages instead. This solves the dropping pages problem,
and doesn't cause even more calls into the cache code.
Thanks to Gleb Natapov for both finding the problem and giving a fix.
This should go to the v1.0 branch
This commit was SVN r8732.
include help-opal-wrapper.txt in the dist tarball directly. We don't want
to include the wrapper-data.txt files, because we actually want to ship the
.in versions of those files, and AM automatically ships aything needed for
configure's output
This commit was SVN r8711.
- fall back to compile test for windows paffinity component
when cross compiling
- fall back to platform guess when checking for threads having
different pids with pthreads (yes on linux, no elsewhere)
- pass the proper host, target, and build flags to the
ROMIO configure script
With these changes, cross-compiling should be possible with the exception
of the Fortran 77 and Fortran 90 bindings. Fortran 77 can be cross-
compiled if cache values are provided for type sizes and alignment.
This commit was SVN r8702.
r8698), with changes below:
- Split wrapper flags into those required for each of the three projects,
and cleaned up some cruft (including the LIBMPI_EXTRA_*FLAGS) through-
out the build system
- Added opal_init_util and opal_finalize_util to allow init / cleanup
of all the opal code that doesn't require the MCA system
- Create standalone key=value file parser, based on the one that used
to be in the mca param parser, so that it can be shared in multiple
places
- Add wrapper datafiles for opal, orte, and ompi wrappers, and add
wrapper compiler with support for all the old features
This commit was SVN r8699.
The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
r8690
r8698
* Make sure --without-BTL works for all BTLs
* Fix copy-n-paste error in aix timer configrue help string
This should go to the v1.0 branch
This commit was SVN r8554.
deallocation came from the allocator (malloc, fee, etc) or somewhere
else (the user calling mmap/munmap, etc). Going to be used by Galen
to determine if it is worth searching the allocations tree
Set flag if it is possible to intercept mmap (not always possible
due to a circular dependency between mmap, dlsym, and calloc)
This commit was SVN r8521.
Jeff-length). Also change it such that we also have a lt_dlhandle
type -- even in the case of --disable-dlopen. Specifically, even in
the case of --disable-dlopen, we still have shell functions that use
an lt_dlhandle parameter, so we need that type, even if <ltdl.h> isn't
used.
This commit was SVN r8507.
- don't fail to configure if threads aren't found, since
we don't have thread support on windows
- rather than setting the asm file name to none in asm-data,
special case windows w/ CL in the ompi_config_asm macros.
Otherwise, there were some warnings during make dist and
configure that didn't need to be there.
This commit was SVN r8502.
- add the right asm format
- add checks for some constants / fields that cygwin
doesn't have in the stacktrace code
- fix for slightly more verbose libtool 2 betas that
have multiple lines for link output
This commit was SVN r8501.
Windows installation provide the 32 and 64 bits atomic operations and compile
only the relevant part. For 64 bits there is a problem, but I already put a
big comment about it in the configure file ...
This commit was SVN r8480.
may call calloc(large number), which causes ptmalloc2 to call mmap, which
causes us to try to dlsym for mmap, which leads to looping badness.
This commit was SVN r8461.
the ptmalloc2 memory hooks component triggers callbacks for memory
allocation / deallocation. If enabled (the default) it is only when
memory is actually obtained from or released to the OS (so little
malloc calls only trigger callbacks if sbrk is called). If disabled,
callbacks are triggered every time malloc/free/etc. is called
* It turns out that syscall and mmap aren't good friends due to the return
type of mmap and some old legacy issues with syscall functions that
take more than 5 parameters. For now, default to either loading
the symbol from glibc using dlsym or using the __m{un,}map functions.
Thanks to George for finding this.
* Fix some dumb typos in the mmap / munmap catching code
This commit was SVN r8410.
both mmap and munmap), adjusting the configure script so that the
component will only be activated on systems that use ptmalloc2 in the
first place -- ie, Linux
* Remove the malloc_hooks component - it became an unworkable solution
once threads and such were considered.
* Remove malloc_interpose component - it never worked quite right and
was not going to be able to intercept malloc, so it wasn't going to
be useful for OMPI's purposes.
* Update tests a little bit to match recent memory hooks api
issues - still needs a bit of work.
This commit was SVN r8381.
* turns out (duh!) that there was a reason that the <projectdir>dir
variable was set in the AM conditional. If not, stupid directories
are created and not needed... duh.
This commit was SVN r8205.
component/base Makefile.am files, reducing the time configure spends
stamping out Makefiles at the end
* Install base_impl.h file when devel-headers are being installed
This commit was SVN r8200.
When compiling C++ code that includes something that looks for the C++
header file "memory" (stupid C++ headers not having .h extensions), it
goes through the header file search path, which includes $(topsrcdir)/opal,
so it finds the directory $(topsrcdir)/opal/memory/ and tries to load
that as the memory header file and all goes downhill.
This commit was SVN r8111.
test isn't really needed to make sure the malloc intercept code was working,
so just revert to not checking for now.
This should go to the 1.0 branch, as what is there now is causing issues.
This commit was SVN r8081.
this function, and the lock will still be held, so we'd have deadlock.
Instead, unlock, then release. We don't need the lock for calling the
release - we're not modifying any shared variables my that point.
This should probbaly go onto the 1.0 branch, although it would be nearly
impossible to trigger the deadlock with the way we use the free handlers
at present.
This commit was SVN r8057.
--with-devel-headers is given to configure:
* allocator, rcache, and mpool were putting things in the wrong place
* timer wasn't installing the inline implementations at all
This commit was SVN r7805.
AC_CHECK_FUNCS, so #ifdef is the correct preprocessor conditional here
(because it will be defined or not -- not always defined to, say, 0 or
1).
This commit was SVN r7781.
originally suggested by Ralf Wildenhues, to try to speed autogen, configure,
and make (and possibly even make install). Use automake's include directive
to drastically reduce the number of Makefile files (although the number of
Makefile.am files is the same - most are just included in a top-level
Makefile.am). Also use an Automake SUBDIRs feature to eliminate the
dynamic-mca tree, which was no longer really needed. This makes adding
a framework easier (since you don't have to remember the dynamic-mca
tree) and makes building faster (as make doesn't have to recurse through
the dynamic-mca tree)
This commit was SVN r7777.
OMPI_CHECK_PACKAGE macro instead of doing everything ourself.
The old code was causing problems - it wouldn't add anything to
WRAPPER_EXTRA_{LDFLAGS, LIBS} if libnuma was installed in /usr,
so it didn't work so well.
This should go to the 1.0 branch
This commit was SVN r7683.
Some of the functions in opal_init are void or return a bool (opal_output_init, but always returns true.. eh?), so I don't check them.
This commit was SVN r7638.
in finalize after the first cntl-c is sent.
If the signal delete function was called while the signal set was blocked (which
is really most of the time in OMPI), the signal handler would be set to default
and the event library sigset modified, but the signal would never be unblocked.
Now we unblock the signal after reseting the handler...
This commit was SVN r7623.
set it to empty if it isn't. __malloc_attribute__ is set in sys/cdef.h, and
some versions set it unconditionally. If sys/cdef.h is included after malloc.h,
some compilers will complain loudly. So include malloc.h at the end so that
sys/cdef.h is already included if it's going to be.
This commit was SVN r7613.
expand it's class list array by pre-allocating the callback item before
we obtain the lock (we free it if we ended up not needing it)
This commit was SVN r7533.
out-of-bounds protection for output ID's in some functions. Also,
move some logic for closing the syslog inside a conditional block --
it's only necessary to close the syslog if we actually closed a
stream.
This commit was SVN r7518.
in-place already (turns out that I was wrong in thinking that it
didn't work for static components), but the logic for excluding
components was not there. This commit does a few things:
- Adds "exclude" logic, so that you can do:
mpirun --mca btl ^mvapi,openib ...
(note the "^" character -- I tried "!" but then you have to escape
it in the shell, and that was icky) which will exclude both the
mvapi and openib btl components (excluding one component means that
you are excluding all components in the list; it doesn't make sense
to include some and exclude others -- you're entire entirely
including or entirely excluding)
- Simplifies the "include" logic, so the same old stuff like this
still works:
mpirun --mca btl tcp,self ...
will only use the tcp and self btl components.
- Added more verbosity statements to make this selection process
clear.
This commit was SVN r7509.
really can't. Test for munmap, since it's the most likely to cause problems,
since it's always an interposed symbol.
The condition that usually causes problems is if libmpi was brought in as
the result of a library dependency, rather than as a -l on the link line.
The linker in this case will find malloc/free/munmap/etc. in libc, rather
than in libmpi.
This commit was SVN r7508.
moving the tick register to another register, but is fine with rd.
* properly mask out the upper 32 bits of the register containing the
lower 32 bits so that the or with the upper 32bits actually works
properly. Hopefully, gcc will optimize this into code that doesn't
suck so much (it's much easier to deal with when you get to control
the argument registers...)
This commit was SVN r7410.
Makefile.options
- Sample in each of the three projects of how to link againt the
relevant libraries so that when components are loaded into a parent
process' space, we don't rely on the libopal/liborte/libmpi symbols
being in the parent's public symbol namespace -- instead,
dynamically link to the relevant libraries, allowing the dynamic
linker to pull those libraries in at run-time, if needed
This commit was SVN r7397.
that multiple processes don't overwrite each other. Change that
default in orte_init_stage1() to just "output-" (because the file will
be in a process-unique directory at that point; the pid is no longer
necessary).
This commit was SVN r7256.
opal_output_set_output_file_info(). This allows getting and setting
the default directory where output stream files will be opened (for
all *new* streams). Before this function is not invoked, the default
location is $TMPDIR or $HOME (if $TMPDIR is not defined).
Added a call into orte_init_stage1() to call this function
immediately after the session directory is created and set the default
location of stream files to be the process' session directory.
This commit was SVN r7254.
a constructor, like the rest of the code base
- Convert usage in the tree to use the constructor to zero out an
instance of opal_output_stream_t
- Still need to re-enable output files
This commit was SVN r7253.
AM_INIT_AUTOMAKE, instead of the deprecated version.
* Work around dumbness in modern AC_INIT that requires the version
number to be set at autoconf time (instead of at configure time, as
it was before). Set the version number, minus the subversion r number,
at autoconf time. Override the internal variables to include the r
number (if needed) at configure time. Basically, the right thing
should always happen. The only place it might not is the version
reported as part of configure --help will not have an r number.
* Since AM_INIT_AUTOMAKE taks a list of options, no need to specify
them in all the Makefile.am files.
* Addes support for subdir-objects, meaning that object files are put
in the directory containing source files, even if the Makefile.am is
in another directory. This should start making it feasible to
reduce the number of Makefile.am files we have in the tree, which
will greatly reduce the time to run autogen and configure.
This commit was SVN r7211.
be called with a description of its memory segments to make local. It
is a small enough API that changing to support a
one-process-does-all-assignment model is simple enough if we ever need
it.
This commit was SVN r7148.
add a -I to find the included ltdl.h (vs. a system-installed ltdl.h)
- Clean up kruft in a bunch of Makefile.am's to remove now-unnecessary
AM_CPPFLAGS settings to get static-components.h for each framework
- Move the component_repository API functions out of opal/mca/base/base.h
and into opal/mca/base/mca_base_component_repository.h in order to
decrease unnecessary dependencies (e.g., before this, almost
everything in the tree depended on ltdl.h, which is unnecessary --
only a small number of files really need ltdl.h)
This commit was SVN r7127.
orte_init_stage1(), since not all ORTE processes call orte_init().
* Expad opal_error test case to make sure ORTE error codes print
properly
* Make project error codes start at easy values (OPAL is -1 to -100,
ORTE is -101 to -200, OMPI is less than -201) to make it easier
to figure out what an error code as an integer means. Also has
the nice property of not changing the values of error codes ever
time a new error code is added.
This commit was SVN r7061.
tree.
- fix up #include's throughout the tree (yay contrib/search_replace.pl!)
- remove a few extraneous #include's
- remove orte_sys_info*() from opal_init()/opal_finalize() (it's
already in orte_init_stage1() and orte_system_finalize())
- remove dependencies in opal on orte_system_info -- util/os_path.c
and util/os_create_dirpath.c (they only used path_sep, anyway --
easily changed to #defines)
This commit was SVN r7059.
API is still a bit unstable and may change.
- Add a primitive "first use" component that simply has each process
"touch" the pages that they want to use, thereby [hopefully] locking
them locally to a specific processor
- Add hooks in ompi_mpi_init to enable memory affinity when processor
affinity is used.
- Added hooks in ompi_mpi_finalize to shut down memory affinity when
it was initialized during ompi_mpi_init.
- Added right hooks in ompi_info to display maffinity components.
This commit was SVN r7044.
prototype than everyone else. It's only active if _GNU_SOURCE is
definted, which of course we have to have defined for asprintf.
So just use strerror() underneath
This commit was SVN r7002.
cast the return to an int in the C++ test case, just in case.
* C++ sucks. If compiling with C++ on some GNU compiler/linker
combos, the initialize hook isn't automagically fired for the
malloc code. Add a backup setting during opal_init, which is
early enough not to cause any damage.
This commit was SVN r6983.
- If passing in a valid argv array with no members (so argv[0] == NULL),
opal_argv_copy() should return a valid argv array with no members,
instead of NULL.
- make sure to update the argc value when deleting members of an array
This commit was SVN r6977.
* make sure LIBS contains -lpmapi before checking for pm_cycles()
* reorder aix functions so that we don't use get_usecs() before we
define it
This commit was SVN r6970.
OPAL_ERROR, same for all the other error codes. Also, make sure that there
are never conflicts between OPAL anr ORTE error codes (for example).
Finally, provide opal_perror(), opal_strerror(), and opal_strerror_r() to
give stringified error messages for the different error codes
This commit was SVN r6969.
* Make ompi_info list timer components
* Remove flag to display whether we have memory intercepts (components are
already listed), until we can figure out how to do it *after* the
components are opened.
This commit was SVN r6950.
on all glibc systems (tested with x86 and x86_64 with a couple of C++
compilers). While not as ideal as the malloc_hooks method, it does
have the advantage of working with threads.
* Modified malloc_hooks component to properly follow prefix rule. No
functionality changes
* Make the memory framework only chose one component, and modify all
components to set priority to 20, except malloc-interpose, which is
at 10. This means that on Linux, malloc_hooks will be used unless
threads are enabled, since I think malloc_hooks is a better design
choice when we can use it
This commit was SVN r6949.
to opal_progress() to use the timers instead of a tick count for deciding
whether to call the event loop or not. Currently supported platforms are:
- solaris (x86 / sparc)
- Linux (x86 / x86_64 / IA64)
- Mac OS X (x86 / Power PC)
This commit was SVN r6922.
* Add memory intercept routines for Darwin using the official Darwin
API (thanks to Drew Gallatin from Myricom for pointing me to some
information from Apple engineers about how to make this work)
* add debugging output to functionality test
This commit was SVN r6920.
all the different cases (curses to RedHat for continually changing
the glibc API!)
- Minor fixes for the Solaris paffinity component
This commit was SVN r6894.
* Add base to memory framework so that we can do something sane with
ompi_info
* Updated ompi_info to print components for memory framework and
show whether we have memory hooks active or not.
This commit was SVN r6861.
- Simple components for getting and setting processor affinity of a
process; does *not* include scheduling decisions
- No one in the OMPI code base invokes the framework yet
- Added linux component for using sched_setaffinity()
- Added shell solaris component that will use processor_bind()
(currently .ompi_ignore'd)
This commit was SVN r6854.
ompi/).
- There's still a handful of places that have orte/ #include files;
still need to clean those up
- A lot of places still use ompi/include/constants.h -- those need to
be converted over to use OPAL_ return codes and then switch to the
opal constants.h. This commit is the first few steps towards
that...
This commit was SVN r6843.
Might want to bump this higher?
* remove assumption that progress functions registered with opal_progress
are not reentrant. There is still the assumption that the event loop
is not reentrant (because it's not), so that part is still protected by
a spinlock.
This commit was SVN r6827.
ompi_config.h in some of the intercept mechanisms.
* Intercept munmap when called directly by the user when we are using
ptmalloc2 (previously we only covered when the user called free()).
* Don't go through the locking and list traversal logic trying to fire
callbacks until there is actually a callback to fire. This is both
a performance boost and a way to cope with the hook callback being
triggered before opal_init.
This commit was SVN r6818.
* don't run callbacks before end of init or after start of finalize, since
the list structures will be in an undefined state at those times.
This commit was SVN r6791.
callbacks to be triggered when memory is about to leave the current
process. The system is designed to allow a variety of interfaces,
hopefully including whole-sale replacement of the memory manager,
ld preload tricks, and hooks into the system memory manager. Since
some of these may or may not be available at runtime and we won't know
until runtime, there is a query funtion to look for availability of
such a setup.
* Added ptmalloc2 memory manager replacement code. Not turned on by
default, can be enabled with --with-memory-manager=ptmalloc2.
Only tested on Linux, not even compiled elsewhere. Do not use
on OS X, or you will never see your process again.
* Added AM_CONDITIONAL for threads test to support ptmalloc2's build
system
This commit was SVN r6790.
that were set on the command line. This was techinically exactly the
way the code was designed, but it certainly violated the Law of Least
Astonishment (even to its designer ;-) ). So now if you execute
something like this:
mpirun -mca pls_rsh_debug 1 -np 4 hello
You'll see debugging output from the rsh pls component, as you would
expect (this was not previously the case -- the MCA pls_rsh_debug
parame would be set to 1 in the 4 spawned hello processes, but *not*
in the orterun process).
More specifically, MCA parameters will be set in the orterun process
in the following cases:
- The new command line switch "--gmca" (or "-gmca") is used,
indicating that the MCA parameter is "global". --gmca also means
that that MCA parameter will be applied to all context app's. For
example:
mpirun -gmca foo bar -np 1 hello : -np 2 goodbye
The foo MCA param will be set in both the hello and goodbye
processes.
- If there is only one context app. For example:
mpirun -mca pls_rsh_debug 1 -np 4 hello
will set pls_rsh_debug to 1 in both the orterun process and the 4
spawned hello processes.
Also added a few more comments inside orterun to document a somewhat
confusing use of a state variable in a recursive case.
This commit was SVN r6764.
properly. This fixes the random hangs that we were seeing this morning
on Linux that were a result of fixing the thread deadlock yesterday.
(worked great on my OS X box, which uses select() instead of poll()).
This commit was SVN r6730.
- new preferred API calls for registering MCA parameters are
mca_base_param_reg_{int|string} and
mca_base_param_reg_{int|string}_name.
- See opal/mca/base/mca_base_param.h for docs on new calls.
- Can now register and lookup a value at the same time.
- Can now mark a parameter "read only" at registration time
- Can now mark a parameter "internal" at registration time
- Can now associate a help message with the parameter at registration
time; displayed in the ompi_info output.
The old API calls are still available for backwards compatibility
(mca_base_param_register_{int|string}. They will eventually be
removed -- all developers are encouraged to use the new APIs from here
on out and replace any old calls with the new API.
Some params were also renamed -- the previous convention of using
"base_" as a prefix for any param that was not associated with a
component is henceforth deprecated. Instead, use one of the following
prefixes:
mca: for anything in the MCA base itself
opal: for anything in OPAL
orte: for anything in ORTE
mpi: for anything in OMPI
This commit was SVN r6698.
- only call sched_yield if it exists
- don't fail out if modex doens't work in ob1
- bunch of fixes for Portals BTL
- add cnos rml component
- add NULL gpr component (should only be used if replica AND proxy
fail to load)
This commit was SVN r6629.
* Add ability to completely disable libltdl (the dlopen code to load
dynamic shared objects) to configure: --disable-dlopen
* Added MCA param (component_disable_dlopen) to disable DSO loading
at runtime
* Made the event library behave in some not-completely-erroneous way
on platforms where it has absolutely no eventops support (ie, no
select, poll, or epoll)
* Disabled orte_wait, opal_few, and opal_daemon_init code on
platforms without fork, waitpid support. All non-init functions
will return OPMI_ERR_NOT_SUPPORTED
* Disable orteprobe tool when fork or pipe aren't supported
This commit was SVN r6490.
Red Storm. Add stub functions to ompi_config_bottom.h when they are
around
* Add protection for a bunch of #include <netinet/in.h>s
* Fix up the Portals BTL so that it compiles on Red Storm and has the
right mojo for initialization on Red Storm
* Add some important comments to ompi_check_package and mvapi configures
* Add support for platforms without getpwuid() (aka, Red Storm).
This commit was SVN r6478.
sockaddr_in - seems to be a good indicator)
* disable util/if code if no inet devices (again, no sockaddr_in)
* add enable/disable flag to disable stacktrace pretty-print code
(defaults to enabled). Seems there's something funky going on with
the preprocessor on Red Storm that was causing problems - this was
the easiest fix
* clean up a bunch of the configure.m4 files to remove bogus comments,
properly comment them, fix the dumb logic for happy/unhappy
* Create a macro for testing both header and library for a package,
since we seem to do this kind of test quite often. Handles the
-I and -L search paths properly (including stripping out /usr and
/usr/local if not needed)
* Converted mvapi components to configure.m4, using the nice new
ompi_check_package macro (above)
This commit was SVN r6454.
an object has been created. I slightly change this behaviour. Now, during the life time of an object (when
the reference count is > 0) we keep the place where the object was created and just before freeing the object
we set the location where the object get destroyed. So when we try to release an already destroyed object
we have the location where it happens (where the object get destroyed for the first time).
This commit was SVN r6428.
* rename ompi_malloc to opal_malloc
* rename ompi_numtostr to opal_numtostr
* start of rename of ompi_environ to opal_environ
This commit was SVN r6332.
* rename ompi_basename to opal_basename
* rename ompi bitop functions to opal
* rename ompi_cmd_line to opal_cmd_line
* rename ompi_sizet2int to opal_sizet2int
* rename orte_daemon_init to opal_daemon_init
* rename ompi_few to opal_few
This commit was SVN r6330.
- move mpool and allocator frameworks back to ompi (from opal)
- specialize the ompi_free_list class to use an mpool instance
- un-specialize opal_free_list to *not* use mpool; just use malloc/free
This commit was SVN r6292.
