1
1
Граф коммитов

862 Коммитов

Автор SHA1 Сообщение Дата
Samuel Gutierrez
c1012f502f added backing file relocation capability in shmem mmap. two new mca
parameters control its behavior (shmem_mmap_relocate_backing_file and
shmem_mmap_backing_file_base_dir).

This commit was SVN r25480.
2011-11-16 01:38:23 +00:00
Ralph Castain
6310361532 At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement

The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.

In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:

1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.

2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.

3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.

As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.

This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
Shiqing Fan
fc46ed6438 Use the new libevent On Windows.
This commit was SVN r25462.
2011-11-09 14:05:35 +00:00
George Bosilca
bd55a19db4 Disable vector optimization for ICC v12.1.0 release 2011.6.233.
This commit was SVN r25461.
2011-11-08 21:23:30 +00:00
Jeff Squyres
78538b701d Turns out that we're not even using that $includedir properly, so just
remove it.

This commit was SVN r25458.
2011-11-08 03:18:09 +00:00
Ralph Castain
a931c5b1eb Redo a patch from late last night that replaces libevent 2.0.7 with libevent 2.0.13 as our default event library. Cleanup the libevent renames to correctly state 2013 as our new version.
This commit was SVN r25457.
2011-11-08 01:36:15 +00:00
Ralph Castain
a3ce355a60 Revert r25453 and r25450 until we can fix the libevent2013 configure code - still not getting the includedir to eval correctly.
This commit was SVN r25454.

The following SVN revision numbers were found above:
  r25450 --> open-mpi/ompi@7f7d5c4f1f
  r25453 --> open-mpi/ompi@c9fe8c32e2
2011-11-07 16:23:44 +00:00
Ralph Castain
c9fe8c32e2 Fix a bug in the configure.m4 to ensure we disable unused libevent modes. Edit their Makefile.am to remove bufferevent support since we don't use it either.
Sorry for mid-day correction.

This commit was SVN r25453.
2011-11-07 15:21:29 +00:00
Nathan Hjelm
7f7d5c4f1f RFC: upgrade to libevent 2.0.13 (removing 2.0.7) timeout. Removed libevent 2.0.7
This commit was SVN r25450.
2011-11-07 04:32:36 +00:00
Jeff Squyres
f08b8bf2d4 Per this thread:
http://www.open-mpi.org/community/lists/devel/2011/10/9878.php

I am making a final decision to decide the behavior of what happens
when an MCA parameter is re-registered and changes types.  In
developer builds (i.e., OPAL_ENABLE_DEBUG==1), a show_help message
will be displayed.  In all builds, an error status will be returned.
Specifically, the logic looks like this:

{{{
    if (detect_re-registration_with_type_change) {
#if OPAL_ENABLE_DEBUG
        opal_show_help(...);
#endif
        return OPAL_ERR_VALUE_OUT_OF_BOUNDS;
    }
}}}

If someone would like to change this behavior, they are welcome to do
so.  :-) I am committing this so that ''some'' action occurs (rather
than talking about the issue and then nothing happens).

This commit was SVN r25432.
2011-11-04 14:16:49 +00:00
Jeff Squyres
886a9d589b Custom patch from Brice for the hwloc-1.2.2ompi distro, per an issue
that Chris Yeoh/IBM found.  See the thread below for more info:

  http://www.open-mpi.org/community/lists/hwloc-devel/2011/11/2521.php

This commit was SVN r25429.
2011-11-03 14:53:22 +00:00
Jeff Squyres
4fe26b0392 Fix some minor memory leaks
This commit was SVN r25410.
2011-11-01 20:22:26 +00:00
Ralph Castain
4368199c86 Missing include
This commit was SVN r25402.
2011-10-31 13:39:57 +00:00
Ralph Castain
96332a2859 Fix typo
This commit was SVN r25400.
2011-10-30 13:23:42 +00:00
Ralph Castain
71ed8e3cd3 Bring back the local node's binding capabilities along with its topology. Clean up indentation.
This commit was SVN r25399.
2011-10-30 13:20:16 +00:00
Ralph Castain
7ba4675adf Bring over some useful utilities and definitions for working with hwloc inside ORTE/OMPI. Cache frequently computed info to save processing time when handling multiple nodes with the same topology. Deal with available cpus as defined by online vs allowed vs user-specified limits. Help deal with hwloc's unfortunate decision to lump all caches in the same object type.
This commit was SVN r25393.
2011-10-29 14:58:58 +00:00
Jeff Squyres
6092b50ebb Fix the cases where the default values of MCA params were not always
handled properly when MCA parameters are re-registered and their types
change.  Specifically, this case was broken:

 1. Register an int MCA param with a non-zero default value
 1. Re-register the same MCA param as a string with a NULL default value

The 2nd step would cause a segv because the first int default value
wasn't being reset properly.  Here's sample code that shows the issue:

{{{
{
    int ibogus;
    char *sbogus;
    opal_init(&argc, &argv);
    mca_base_param_reg_int_name("type", "name", "help", false, false, 3, &ibogus);
    printf("Ibogus: %d\n", ibogus);
    mca_base_param_reg_string_name("type", "name", "help", false, false, NULL, &sbogus);
    printf("Sbogus: %s\n", (NULL == sbogus) ? "NULL" : sbogus);
    exit(0);
}
}}}

This commit fixes the problem from the sample code above as well as
the a similar issue for file-set MCA params and override values.  It
also resets default values for MCA params initially registered as a
string but then re-registered as an int.

This commit was SVN r25392.
2011-10-29 12:29:31 +00:00
Ralph Castain
21d45b0807 Just some cleanup in case of error
This commit was SVN r25387.
2011-10-29 01:55:19 +00:00
Ralph Castain
a7cbc25658 Minor cleanups - check hwloc returns everywhere. Thanks to Chris Yeoh for pointing this out.
This commit was SVN r25360.
2011-10-24 14:05:26 +00:00
Shiqing Fan
5711414eb7 Fix Windows build
This commit was SVN r25351.
2011-10-21 14:46:58 +00:00
Jeff Squyres
cbafea8f69 Add a DEPENDENCIES line so that if you edit something down in the
hwloc tree, it'll get picked up by the component (and therefore by
libopen-pal).

Thanks to Terry for finding the problem.

This commit was SVN r25349.
2011-10-21 11:39:52 +00:00
Ralph Castain
b44f8d4b28 Complete implementation of the ess.proc_get_locality API. Up to this point, the API was only capable of telling if the specified proc was sharing a node with you. However, the returned value was capable of telling you much more detailed info - e.g., if the proc shares a socket, a cache, or numa node. We just didn't have the data to provide that detail.
Use hwloc to obtain the cpuset for each process during mpi_init, and share that info in the modex. As it arrives, use a new opal_hwloc_base utility function to parse the value against the local proc's cpuset and determine where they overlap. Cache the value in the pmap object as it may be referenced multiple times.

Thus, the return value from orte_ess.proc_get_locality is a 16-bit bitmask that describes the resources being shared with you. This bitmask can be tested using the macros in opal/mca/paffinity/paffinity.h

Locality is available for all procs, whether launched via mpirun or directly with an external launcher such as slurm or aprun.

This commit was SVN r25331.
2011-10-19 20:18:14 +00:00
Rainer Keller
ec6ac33b75 - On Linux x86-64 with intel compiler v12.1, any ompi-app fails before
calling main():
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
xxx:~/openmpi-1.5.4/COMPILE-intel-12.1.0> which ompi_info
~/openmpi-1.5.4/COMPILE-intel-12.1.0/usr/bin/ompi_info
xxx:~/openmpi-1.5.4/COMPILE-intel-12.1.0> ompi_info
Segmentation fault
xxx:~/openmpi-1.5.4/COMPILE-intel-12.1.0> gdb usr/bin/ompi_info
...
(gdb) run
Starting program:
...
Program received signal SIGSEGV, Segmentation fault.
opal_memory_ptmalloc2_int_malloc (av=0x7ffff7fe83d8, bytes=4102) at
../../../../../opal/mca/memory/linux/malloc.c:4080
4080          /* remove from unsorted list */
(gdb) where
#0  opal_memory_ptmalloc2_int_malloc (av=0x7ffff7fe83d8, bytes=4102) at
../../../../../opal/mca/memory/linux/malloc.c:4080
#1  0x00007ffff7c232b9 in opal_memory_linux_malloc_hook
(sz=140737354040280, caller=0x1006) at
../../../../../opal/mca/memory/linux/hooks.c:687
#2  0x0000003dd96a6871 in __alloc_dir () from /lib64/libc.so.6
#3  0x0000003ddfa053cd in ?? () from /usr/lib64/libnuma.so.1
#4  0x0000003dd8e0e445 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
...
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    A lot of combinations and trials have been done, yet to no avail.
    Intel v11.0 worked...

    Thanks to Hubert Haberstock (Intel) providing the hint in:
    http://software.intel.com/en-us/forums/showthread.php?t=87132

    This was tested on openmpi-1.5.4 and therefore should
    cmr:v1.5

This commit was SVN r25290.
2011-10-14 20:47:08 +00:00
Ralph Castain
69a0882207 Correctly setup hwloc when passing a topology from an external source
This commit was SVN r25277.
2011-10-12 21:34:46 +00:00
Jeff Squyres
ff97b57c90 Change the names to be slightly more descriptive.
This commit was SVN r25271.
2011-10-12 16:07:09 +00:00
Jeff Squyres
951c745590 We always have hwloc xml support (now that it's built into to hwloc
without needing libxml2).  So OPAL_HAVE_HWLOC_XML is no longer
necessary.  

This commit was SVN r25263.
2011-10-11 20:20:59 +00:00
Jeff Squyres
e4f8b662a1 Remove this component; it was wholly superceded by hwloc122ompi a
little while ago.

This commit was SVN r25261.
2011-10-11 20:13:33 +00:00
Jeff Squyres
7fd3a7f696 Remove no-longer-true comment. Process-wide memory affinity policy is
set by calling opal_hwloc_base_set_process_membind_policy().

This commit was SVN r25260.
2011-10-11 20:00:45 +00:00
Ralph Castain
8c4512a994 Fix the verbose output for caches (again) so they are properly labeled, pending adoption of the upstream patch we supplied.
This commit was SVN r25251.
2011-10-11 05:54:26 +00:00
Swen Boehm
08b4322a1a patched the lex files to not issue the following compiler warning:
'yyunput' defined but not used

This commit was SVN r25246.
2011-10-10 18:13:04 +00:00
George Bosilca
649af6c925 Enumerated mixed with another type (int) is tolerated but
easily fixable.

This commit was SVN r25241.
2011-10-09 03:54:52 +00:00
George Bosilca
9d68d7c0c8 iFix a bunch of warnings.
This commit was SVN r25227.
2011-10-03 18:46:49 +00:00
George Bosilca
b4c076ad28 Remove an unused function.
This commit was SVN r25226.
2011-10-03 18:46:27 +00:00
Jeff Squyres
34deb0db97 Sync with final hwloc 1.2.2 release
This commit was SVN r25221.
2011-10-03 14:12:38 +00:00
Jeff Squyres
6a32aa4a04 Oops -- it looks like we ''do'' still use this variable in the
trunk... 

This commit was SVN r25203.
2011-09-28 12:12:37 +00:00
Jeff Squyres
bc3e213a69 After fixing an svn/hg kerfluffle, there's a few files left over from
last night's hwloc/paffinity/maffinity minor update.  Nothing huge;
just a little cleanup.

This commit was SVN r25202.
2011-09-28 11:46:28 +00:00
Jeff Squyres
9fa2130cfb Fix typo that prevents VPATH builds.
This commit was SVN r25201.
2011-09-28 11:29:12 +00:00
Jeff Squyres
970a75a7b6 Update to a custom OMPI roll of hwloc v1.2.2. Upgrade the configry to
match similar stuff in the event framework; only add CPPFLAGS /
LDFLAGS / LIBS / and WRAPPER_EXTRA_* of the same for the one, single,
winning component (because this framework is compile-time,
one-of-many).

This commit was SVN r25199.
2011-09-27 23:54:09 +00:00
Jeff Squyres
3d61d0f357 Fix up some long-latency bugs in the MCA even framework configury that
only became evident when there was more than one event component.

The libevent2013 component is still ompi_ignore'd for most developers.

This commit was SVN r25198.
2011-09-27 23:18:07 +00:00
Jeff Squyres
d4603f080d Refs trac:2854.
Since hwloc has a dynamic bitmap size, it could actually have bits set
that will not fit in the paffinity mask.  We already made sure that we
didn't overrun the paffinity mask; now also set the return value to
OPAL_ERR_VALUE_OUT_OF_BOUNDS (wow, we really thought of everything
with those error codes, eh?) if the hwloc bitmap has bits set higher
than what will fit into the paffinity bitmask.

This commit was SVN r25179.

The following Trac tickets were found above:
  Ticket 2854 --> https://svn.open-mpi.org/trac/ompi/ticket/2854
2011-09-24 13:52:27 +00:00
Jeff Squyres
57323570e3 These calls were mistakenly added in r25164.
This commit was SVN r25176.

The following SVN revision numbers were found above:
  r25164 --> open-mpi/ompi@51129cc2a8
2011-09-22 18:20:30 +00:00
Jeff Squyres
82c93611e6 Fix some problems with the libevent and hwloc frameworks:
* change components from setting <framework>_base_include to
   opal_<framework>_<component>_include; the framework m4 will figure
   out the winning component and pick the right "include" shell
   variable.  Ditto for the other shell variables (cppflags, ldflags,
   etc.). 
 * misc fixes to hwloc/external
 * add a bunch of missing "opal_" prefixes to shell variables
 * add a few more / update a few comments in framework m4's

This commit was SVN r25174.
2011-09-21 23:06:13 +00:00
Ralph Castain
a818101d19 Silence compiler warning
This commit was SVN r25172.
2011-09-21 16:39:59 +00:00
Shiqing Fan
3e0ee394ef Select only one libevent component.
This commit was SVN r25169.
2011-09-20 16:10:01 +00:00
Shiqing Fan
4caed984ed Need to exclude another file for windows build.
This commit was SVN r25168.
2011-09-20 16:09:03 +00:00
Ralph Castain
51129cc2a8 If built without hwloc xml support, we cannot currently pass the local topology from the daemon to an MPI app. This makes it impossible to set affinity, for example. In this case, have the app get its own copy of the topology at startup.
For safety sake, protect hwloc-based affinity modules from NULL topology

This commit was SVN r25164.
2011-09-20 14:46:55 +00:00
Ralph Castain
052ccd4b1e Set ignores
This commit was SVN r25163.
2011-09-20 13:47:05 +00:00
Ralph Castain
45396d8f9c Add missing files
This commit was SVN r25162.
2011-09-20 13:37:22 +00:00
Nathan Hjelm
7cd8f21b7f add libevent 2.0.13 module
This commit was SVN r25161.
2011-09-20 00:13:05 +00:00
Jeff Squyres
9db4542c2b Move maffinity_base_alloc_policy and
maffinity_base_bind_failure_action MCA params to the hwloc base
(hwloc_base_alloc_polocy and hwloc_base_bind_failure_action).  Since
these MCA parameters were never on a release branch, I'm just
moving/renaming them outright and not leaving aliases to the old
names.

Note that some upper layer needs to call
opal_hwloc_base_set_process_membind_policy() to set the
set-by-MCA-param process-wide memory affinity policy.  We can't do
this automatically during hwloc_base_open() because, for reasons
described elsewhere, opal_hwloc_topology is not automatically filled
during hwloc_base_open() (in short: potential scalability issues when
launching many MPI processes simultaneously on a single machine, for
example).

This commit was SVN r25156.
2011-09-19 16:10:37 +00:00