1
1
Граф коммитов

162 Коммитов

Автор SHA1 Сообщение Дата
Ralph Castain
75c306994e Add some debug
This commit was SVN r29523.
2013-10-26 02:26:21 +00:00
Ralph Castain
772a376d73 Correct location of elog file
Refs trac:3847

This commit was SVN r29438.

The following Trac tickets were found above:
  Ticket 3847 --> https://svn.open-mpi.org/trac/ompi/ticket/3847
2013-10-14 19:21:45 +00:00
Ralph Castain
24c811805f ****************************************************************
This change contains a non-mandatory modification
       of the MPI-RTE interface. Anyone wishing to support
       coprocessors such as the Xeon Phi may wish to add
       the required definition and underlying support
****************************************************************

Add locality support for coprocessors such as the Intel Xeon Phi.

Detecting that we are on a coprocessor inside of a host node isn't straightforward. There are no good "hooks" provided for programmatically detecting that "we are on a coprocessor running its own OS", and the ORTE daemon just thinks it is on another node. However, in order to properly use the Phi's public interface for MPI transport, it is necessary that the daemon detect that it is colocated with procs on the host.

So we have to split the locality to separately record "on the same host" vs "on the same board". We already have the board-level locality flag, but not quite enough flexibility to handle this use-case. Thus, do the following:

1. add OPAL_PROC_ON_HOST flag to indicate we share a host, but not necessarily the same board

2. modify OPAL_PROC_ON_NODE to indicate we share both a host AND the same board. Note that we have to modify the OPAL_PROC_ON_LOCAL_NODE macro to explicitly check both conditions

3. add support in opal/mca/hwloc/base/hwloc_base_util.c for the host to check for coprocessors, and for daemons to check to see if they are on a coprocessor. The former is done via hwloc, but support for the latter is not yet provided by hwloc. So the code for detecting we are on a coprocessor currently is Xeon Phi specific - hopefully, we will find more generic methods in the future.

4. modify the orted and the hnp startup so they check for coprocessors and to see if they are on a coprocessor, and have the orteds pass that info back in their callback message. Automatically detect that coprocessors have been found and identify which coprocessors are on which hosts. Note that this algo isn't scalable at the moment - this will hopefully be improved over time.

5. modify the ompi proc locality detection function to look for coprocessor host info IF the OMPI_RTE_HOST_ID database key has been defined. RTE's that choose not to provide this support do not have to do anything - the associated code will simply be ignored.

6. include some cleanup of the hwloc open/close code so it conforms to how we did things in other frameworks (e.g., having a single "frame" file instead of open/close). Also, fix the locality flags - e.g., being on the same node means you must also be on the same cluster/cu, so ensure those flags are also set.

cmr:v1.7.4:reviewer=hjelmn

This commit was SVN r29435.
2013-10-14 16:52:58 +00:00
Ralph Castain
e01953b440 Per Brice, silence warning on old Linux kernels
Refs trac:3744

This commit was SVN r29179.

The following Trac tickets were found above:
  Ticket 3744 --> https://svn.open-mpi.org/trac/ompi/ticket/3744
2013-09-16 15:43:33 +00:00
Ralph Castain
845e92bc5d Remove the old version of hwloc. Update the new one to reflect the official release dates.
Refs trac:3744

This commit was SVN r29154.

The following Trac tickets were found above:
  Ticket 3744 --> https://svn.open-mpi.org/trac/ompi/ticket/3744
2013-09-10 16:30:13 +00:00
Ralph Castain
46ed907003 Correctly handle list of cores specified in the rankfile - i.e., a rankfile entry such as:
rank 0=foo slot=0:0-1;1:0,1

cmr:v1.7.3:reviewer=jsquyres

This commit was SVN r29152.
2013-09-08 02:04:29 +00:00
Ralph Castain
0d7fb932f1 Remove build product file
Refs trac:3744

This commit was SVN r29120.

The following Trac tickets were found above:
  Ticket 3744 --> https://svn.open-mpi.org/trac/ompi/ticket/3744
2013-09-04 16:38:22 +00:00
Ralph Castain
6011a4d29c As per the telecon, update hwloc to v1.7.2 so we can add MIC support. Ignore hwloc1.5.2 component for now until this tests out - will remove it then.
cmr:v1.7.4:reviewer=jsquyres

This commit was SVN r29107.
2013-09-03 16:23:42 +00:00
Ralph Castain
7a7cfdd519 A little cleanup - the base function to sort numa lists must return something or you get a warning about non-void function returning without value, so cleanup the return values. Ensure the mindist module actually checks for a return of "error" so it won't segfault, and have it emit a polite message when that happens.
cmr:v1.7.3:reviewer=jladd

This commit was SVN r29089.
2013-08-29 20:01:06 +00:00
Joshua Ladd
1802aabf1a Add support for autodetecting a MLNX HCA in the rmaps min distance feature. In this way, .ini files distributed with software stacks need not specify a particular HCA but instead may select the key word auto which will automatically select the discovered device. To use this feature, simply pass the keyword auto instead of a specific device name, --mca rmaps_base_dist_hca auto. If more than one card is installed, the mapper will inform the user of this and, at this point, the user will then need to specify which card via the normal route, e.g. --mca rmaps_base_dist_hca <dev_name>. This should be added to \ncmr=v1.7.4:reviewer=rhc:subject=Autodetect logic for min dist mapping
This commit was SVN r29079.
2013-08-28 16:23:33 +00:00
Ralph Castain
446e33a5d8 There are cases where we want to use the novm state machine, but the backend node topology differs from that where mpirun is executing. In those cases, we can wind up thinking we are oversubscribed because the head node has fewer cores than the compute nodes.
To resolve this situation, add the ability to specify a backend topology file that mpirun shall use for its mapping operations. Create a new "set_topology" function in opal hwloc to support it.

This commit was SVN r28682.
2013-06-27 03:04:50 +00:00
Joshua Ladd
46362d2761 Stomps compiler warnings in HCA min-dist calculation. This should be added to cmr:v1.7:reviewer=jladd
This commit was SVN r28620.
2013-06-12 16:25:25 +00:00
Jeff Squyres
6d173af329 This commit introduces a new "mindist" ORTE RMAPS mapper, as well as
some relevant updates/new functionality in the opal/mca/hwloc and
orte/mca/rmaps bases.  This work was mainly developed by Mellanox,
with a bunch of advice from Ralph Castain, and some minor advice from
Brice Goglin and Jeff Squyres.

Even though this is mainly Mellanox's work, Jeff is committing only
for logistical reasons (he holds the hg+svn combo tree, and can
therefore commit it directly back to SVN).

-----

Implemented distance-based mapping algorithm as a new "mindist"
component in the rmaps framework.  It allows mapping processes by NUMA
due to PCI locality information as reported by the BIOS - from the
closest to device to furthest.

To use this algorithm, specify:

   {{{mpirun --map-by dist:<device_name>}}}

where <device_name> can be mlx5_0, ib0, etc.

There are two modes provided:

 1. bynode: load-balancing across nodes
 1. byslot: go through slots sequentially (i.e., the first nodes are
     more loaded)

These options are regulated by the optional ''span'' modifier; the
command line parameter looks like:

    {{{mpirun --map-by dist:<device_name>,span}}}

So, for example, if there are 2 nodes, each with 8 cores, and we'd
like to run 10 processes, the mindist algorithm will place 8 processes
to the first node and 2 to the second by default. But if you want to
place 5 processes to each node, you can add a span modifier in your
command line to do that.

If there are two NUMA nodes on the node, each with 4 cores, and we run
6 processes, the mindist algorithm will try to find the NUMA closest
to the specified device, and if successful, it will place 4 processes
on that NUMA but leaving the remaining two to the next NUMA node.

You can also specify the number of cpus per MPI process. This option
is handled so that we map as many processes to the closest NUMA as we
can (number of available processors at the NUMA divided by number of
cpus per rank) and then go on with the next closest NUMA.

The default binding option for this mapping is bind-to-numa. It works
if you don't specify any binding policy. But if you specified binding
level that was "lower" than NUMA (i.e hwthread, core, socket) it would
bind to whatever level you specify.

This commit was SVN r28552.
2013-05-22 13:04:40 +00:00
Jeff Squyres
55382c1bf8 Bring over upstream hwloc trunk commit
https://svn.open-mpi.org/trac/hwloc/changeset/5592 to fix the merging
of groups when they are I/O objects.

This commit was SVN r28551.
2013-05-22 12:34:59 +00:00
Jeff Squyres
4b9b3a81ff Update the list of post-1.5.2 r numbers from hwloc that we have
committed here.

This commit was SVN r28458.
2013-05-07 01:22:06 +00:00
Jeff Squyres
ee0cdf86fd Fix issue raised by Stefan Friedel: remove an extraneous -L that is
added by hwloc's embedding so that it doesn't appear in
libhwloc_embedded.la (and therefore propogate all the way up to
libmpi.la). 

Committed upstream in hwloc SVN r5588.

This commit was SVN r28457.

The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
  r5588
2013-05-07 01:21:18 +00:00
Ralph Castain
c081a520a3 Fix --without-hwloc
This commit was SVN r28396.
2013-04-25 19:13:56 +00:00
Jeff Squyres
349ee654c1 Fix some --without-hwloc compile errors. Also remove one
assigned-but-not-used variable assignment.

This commit was SVN r28321.
2013-04-10 15:08:31 +00:00
Ralph Castain
3bfa53eb91 Cleanup (again) the solaris topology code in hwloc...sigh.
This commit was SVN r28294.
2013-04-06 14:45:32 +00:00
Ralph Castain
ec00fa3132 Fix missing variable declaration in hwloc 1.5.2
This commit was SVN r28293.
2013-04-05 17:43:34 +00:00
Nathan Hjelm
365cf48db5 Update OPAL frameworks to use the MCA framework system.
This commit was SVN r28239.
2013-03-27 21:11:47 +00:00
Nathan Hjelm
cf377db823 MCA/base: Add new MCA variable system
Features:
 - Support for an override parameter file (openmpi-mca-param-override.conf).
   Variable values in this file can not be overridden by any file or environment
   value.
 - Support for boolean, unsigned, and unsigned long long variables.
 - Support for true/false values.
 - Support for enumerations on integer variables.
 - Support for MPIT scope, verbosity, and binding.
 - Support for command line source.
 - Support for setting variable source via the environment using
   OMPI_MCA_SOURCE_<var name>=source (either command or file:filename)
 - Cleaner API.
 - Support for variable groups (equivalent to MPIT categories).

Notes:
 - Variables must be created with a backing store (char **, int *, or bool *)
   that must live at least as long as the variable.
 - Creating a variable with the MCA_BASE_VAR_FLAG_SETTABLE enables the use of
   mca_base_var_set_value() to change the value.
 - String values are duplicated when the variable is registered. It is up to
   the caller to free the original value if necessary. The new value will be
   freed by the mca_base_var system and must not be freed by the user.
 - Variables with constant scope may not be settable.
 - Variable groups (and all associated variables) are deregistered when the
   component is closed or the component repository item is freed. This
   prevents a segmentation fault from accessing a variable after its component
   is unloaded.
 - After some discussion we decided we should remove the automatic registration
   of component priority variables. Few component actually made use of this
   feature.
 - The enumerator interface was updated to be general enough to handle
   future uses of the interface.
 - The code to generate ompi_info output has been moved into the MCA variable
   system. See mca_base_var_dump().

opal: update core and components to mca_base_var system
orte: update core and components to mca_base_var system
ompi: update core and components to mca_base_var system

This commit also modifies the rmaps framework. The following variables were
moved from ppr and lama: rmaps_base_pernode, rmaps_base_n_pernode,
rmaps_base_n_persocket. Both lama and ppr create synonyms for these variables.

This commit was SVN r28236.
2013-03-27 21:09:41 +00:00
Jeff Squyres
1a048d6ee6 Remove a duplicate variable declaration.
This commit was SVN r28224.
2013-03-27 01:15:27 +00:00
Ralph Castain
317915225c Finish the binding cleanup by removing the no-longer-used binding level scheme. This proved to be fallible as there is no guarantee that the hierarchy it used matched physical reality of the machine (e.g., is L3 "above" the socket or not). Still have to complete the ppr update, but get the rest of it correct.
This commit was SVN r28223.
2013-03-26 20:09:49 +00:00
Ralph Castain
6ee32767d4 Restore the cpus-per-proc option for byslot and bynode mapping. Remove the bind_idx (which recorded the index of the hwloc object where the proc was bound) as this would no longer be unique, and just use the bitmap as the standard reference for location. Update the relative locality computation to take bitmaps as its argument.
This commit was SVN r28219.
2013-03-26 18:27:50 +00:00
Jeff Squyres
6c8d0450a3 Update the post-hwloc-1.5.2 patch list.
This commit was SVN r28218.
2013-03-26 16:18:52 +00:00
Jeff Squyres
f79716dfd4 Include <hwloc.h> so that the symbols in this file are subject to the
<hwloc/rename.h> renaming.

This commit was SVN r28215.
2013-03-26 15:49:52 +00:00
Jeff Squyres
6695b5e17a Re-apply r28040 from Eugene: a post-hwloc release fix for Solaris
binding.  This fix was included in the upstream 1.6 series, but not
the upstream 1.5 series, and was therefore missed when we brought
1.5.2 to OMPI.

This commit was SVN r28212.

The following SVN revision numbers were found above:
  r28040 --> open-mpi/ompi@3d44f97572
2013-03-26 13:27:23 +00:00
Ralph Castain
8a79d37ac2 Fix a few bugs in the hwloc integration code. The "set binding policy" macro should flag that the policy was indeed set. Some systems don't report sockets, so the print functions need to check for that condition.
cmr:v1.7

This commit was SVN r28209.
2013-03-25 17:51:45 +00:00
Jeff Squyres
e5838e6121 Don't mandate PCI support, because this will make builds on platforms
that don't have libpciaccess fail (e.g., OS X, or any machine without
libpciaccess).

This commit was SVN r28181.
2013-03-19 16:20:08 +00:00
Jeff Squyres
90802410a8 Update hwloc from 1.5.1 to 1.5.2. Re-enable hwloc PCI support by
default, since it will now use libpciaccess (if available).

This commit was SVN r28178.
2013-03-18 23:02:56 +00:00
Ralph Castain
8d2fa3693b First cut at removing the native Windows support. Remove all the Windows-specific components, and the .windows files sprinkled around. Remove the Windows platform files and MTT scripts. Update the NEWS to point Windows users to the cygwin package.
This commit was SVN r28116.
2013-02-26 20:44:56 +00:00
Brian Barrett
7c3e42a689 Work around issue shown in #3505 by not linking against libpci by default.
This commit was SVN r28076.
2013-02-19 16:19:33 +00:00
Ralph Castain
037918e7b4 Correctly parse the rank file slot_list when given "S:C" - the first position holds the socket, so start looking for cores at posn=1
This commit was SVN r28054.
2013-02-13 13:06:03 +00:00
Eugene Loh
3d44f97572 Fix hwloc get-cpubind routine for Solaris. FIRST, check
processor_bind to see if we're bound to a single core.
If not, THEN check lgroup affinity.  Already CMR'ed to
v1.6 (trac 3507) and fixed upstream in hwloc (r5295).

This commit was SVN r28040.

The following SVN revision numbers were found above:
  r5295 --> open-mpi/ompi@6df8cb0f02
2013-02-10 04:02:19 +00:00
Brian Barrett
b8442ba505 Revamp the handling of wrapper compiler flags. The user flags, main configure
flags, and mca flags are kept seperate until the very end.  The main configure
wrapper flags should now be modified by using the OPAL_WRAPPER_FLAGS_ADD
macro.  MCA components should either let <framework>_<component>_{LIBS,LDFLAGS}
be copied over OR set <framework>_<component>_WRAPPER_EXTRA_{LIBS,LDFLAGS}.
The situations in which WRAPPER CPPFLAGS can be set by MCA components was
made very small to match the one use case where it makes sense.

This commit was SVN r27950.
2013-01-29 00:00:43 +00:00
Ralph Castain
f6b4db0b79 Fix rank_file operations. We changed the syntax to use semi-colons between multiple slot assignments so that we could use the comma to separate specific cores, but somehow the flex definitions didn't get updated to accept that character. We also incorrectly zero'd the bitmap between slot assignment sections, and so multiple slot assignments only wound up making the last one in the list.
This commit was SVN r27908.
2013-01-25 18:33:25 +00:00
Brian Barrett
4f41f5ce5b OPAL_WRAPPER_EXTRA_CPPFLAGS is the wrong variable, want to set
WRAPPER_EXTRA_CPPFLAGS

This commit was SVN r27886.
2013-01-21 23:37:35 +00:00
Ralph Castain
2379b7369f Hey Jeff - AC_HELP_STRING takes *two* arguments, dude!
This commit was SVN r27820.
2013-01-15 15:25:58 +00:00
Jeff Squyres
e30d9a2bfb The "external" hwloc component didn't have the same fixes applied to
it that the others did: move the "I won!" code up into the POST_CONFIG
macro.  Also, fix a long-standing typo when restoring the $CPPFLAGS (!).

This commit was SVN r27813.
2013-01-14 21:44:47 +00:00
Jeff Squyres
423208932e HWLOC_DO_AM_CONDITIONALS must be run unconditionally.
This commit was SVN r27812.
2013-01-14 21:43:16 +00:00
Jeff Squyres
c17ec83de3 Add some post-v1.5.1 release hwloc bug fixes
This commit was SVN r27805.
2013-01-14 16:25:21 +00:00
Jeff Squyres
c7cb363da9 Remove some more generated files.
This commit was SVN r27800.
2013-01-12 03:30:43 +00:00
Jeff Squyres
4d6f026941 Fix a typo.
This commit was SVN r27799.
2013-01-12 03:30:29 +00:00
Ralph Castain
4d43585a1e Cleanup new hwloc install - remove build products that were accidentally included in the commit, remove non-existent file from Makefile.am
This commit was SVN r27798.
2013-01-12 03:21:53 +00:00
Jeff Squyres
3ce170d463 Update the embedded hwloc from v1.4.2 to v1.5.1.
This commit was SVN r27797.
2013-01-12 02:08:04 +00:00
Jeff Squyres
427c154800 Similar to r27794, simplify the hwloc framework by changing it to
STOP_AT_FIRST.  And move the side-effect-inducing code in
hwloc142/configure.m4 up to POST_CONFIG.

Also change the priority of the external hwloc component to 90 so that
it is evaluated before the internal component (as a direct result of
changing to STOP_AT_FIRST).

This commit was SVN r27796.

The following SVN revision numbers were found above:
  r27794 --> open-mpi/ompi@569a60c2de
2013-01-12 01:48:53 +00:00
Nathan Hjelm
87e5f97400 add missing #include of opal/util/output.h
This commit was SVN r27599.
2012-11-13 07:14:41 +00:00
Nathan Hjelm
bdedd8b0d3 Per RFC modify the behavior of mca_base_components_close to NOT close the output. Modify frameworks to always close their output and set to -1.
Reasoning: The old behavior was a little confusing. mca_base_components_open does not open an output stream so it is a little unexpected that mca_base_components_close does. To add to this several frameworks (that don't use mca_base_components_close) failed to close their output in the framework close function and others closed their output a second time. This change is an improvement to the symantics of mca_base_components_open/close as they are now symetric in their functionality.

This commit was SVN r27570.
2012-11-06 19:09:26 +00:00
Jeff Squyres
6af6809dc2 * Fix some comments.
* Use the hwloc logical index, not the os_index.  Fixes problems with
   opal_hwloc_base_cset2str() output (e.g., --report-bindings output)
   on machines where the os_index is not tightly packed in the range
   ![0, n-1]

This commit was SVN r27394.
2012-10-03 09:33:40 +00:00