for the SM and TCP BTLs, as well as the mca_btl_base_param_register()
function (which registers MCA params for all BTLs).
The guidelines in
https://svn.open-mpi.org/trac/ompi/wiki/MCAParamLevels were used to
pick these levels.
This commit was SVN r28746.
Notes:
- This commit also eliminates the need for an available components list in use
in several frameworks. None of the code in question was making use of the
priority field of the priority component list item so these extra lists were
removed.
- Cleaned up selection code in several frameworks to sort lists using opal_list_sort.
- Cleans up the ompi/orte-info functions. Expose the functions that construct the
list of params so they can be used elsewhere.
patches for mtl/portals4 from brian
missed a few output variables in openib
This commit was SVN r28241.
Features:
- Support for an override parameter file (openmpi-mca-param-override.conf).
Variable values in this file can not be overridden by any file or environment
value.
- Support for boolean, unsigned, and unsigned long long variables.
- Support for true/false values.
- Support for enumerations on integer variables.
- Support for MPIT scope, verbosity, and binding.
- Support for command line source.
- Support for setting variable source via the environment using
OMPI_MCA_SOURCE_<var name>=source (either command or file:filename)
- Cleaner API.
- Support for variable groups (equivalent to MPIT categories).
Notes:
- Variables must be created with a backing store (char **, int *, or bool *)
that must live at least as long as the variable.
- Creating a variable with the MCA_BASE_VAR_FLAG_SETTABLE enables the use of
mca_base_var_set_value() to change the value.
- String values are duplicated when the variable is registered. It is up to
the caller to free the original value if necessary. The new value will be
freed by the mca_base_var system and must not be freed by the user.
- Variables with constant scope may not be settable.
- Variable groups (and all associated variables) are deregistered when the
component is closed or the component repository item is freed. This
prevents a segmentation fault from accessing a variable after its component
is unloaded.
- After some discussion we decided we should remove the automatic registration
of component priority variables. Few component actually made use of this
feature.
- The enumerator interface was updated to be general enough to handle
future uses of the interface.
- The code to generate ompi_info output has been moved into the MCA variable
system. See mca_base_var_dump().
opal: update core and components to mca_base_var system
orte: update core and components to mca_base_var system
ompi: update core and components to mca_base_var system
This commit also modifies the rmaps framework. The following variables were
moved from ppr and lama: rmaps_base_pernode, rmaps_base_n_pernode,
rmaps_base_n_persocket. Both lama and ppr create synonyms for these variables.
This commit was SVN r28236.
ompi_show_help, because opal_show_help is replaced with an
aggregating version when using ORTE, so there's no reason to
directly call orte_show_help.
This commit was SVN r28051.
Update all the orte ess components to remove their associated APIs for retrieving proc data. Update the grpcomm API to reflect transfer of set/get modex info to the db framework.
Note that this doesn't recreate the old GPR. This is strictly a local db storage that may (at some point) obtain any missing data from the local daemon as part of an async methodology. The framework allows us to experiment with such methods without perturbing the default one.
This commit was SVN r26678.
Roll in the ORTE state machine. Remove last traces of opal_sos. Remove UTK epoch code.
Please see the various emails about the state machine change for details. I'll send something out later with more info on the new arch.
This commit was SVN r26242.
specify btl_tcp_if_include because btl_tcp_if_exclude is defaulted to
the loopback devices.
This commit does a few things:
* Introduce a new OPAL MCA base function:
mca_base_param_check_exclusive_string(). It checks to see that the
''user'' does not set two MCA parameters that are mutually
exclusive by checking the source of those MCS param values.
* Use the above function in many BTLs (and the OOB TCP) to ensure
that <foo>_if_include and <foo>_if_exclude are not both specified
''by the user''.
* Re-arrange many of these BTLs to move their MCA registration code
into a separate component_register() function (vs. the
component_open() function).
This code has been nominally reviewed and checked by Ralph, George,
Terry, and Shiqing.
This commit was SVN r25043.
The following SVN revision numbers were found above:
r24976 --> open-mpi/ompi@8f4ac54336
btl_tcp_if_include and btl_tcp_if_exclude are specified.
This commit was SVN r24976.
The following Trac tickets were found above:
Ticket 2838 --> https://svn.open-mpi.org/trac/ompi/ticket/2838
If specified, a comma-delimited list of TCP interfaces. Interfaces
will be assigned, one to each MPI process, in a round-robin fashion
on each server. For example, if the list is "eth0,eth1" and four
MPI processes are run on a single server, then local ranks 0 and 2
will use eth0 and local ranks 1 and 3 will use eth1.
This feature is only useful for environments with virtual ethernet
interfaces on the same network. For example, if eth0 and eth1 are
virtual interfaces to the same NIC on the same subnet, and if the NIC
provides different hardware resources to eth0 and eth1 (not just
different kernel resources), some HOL blocking and congestion issues
can be eased in a modest fashion.
This commit was SVN r24181.
Note: the ompi_check_libfca.m4 file had to be modified to avoid it stomping on global CPPFLAGS and the like. The file was also relocated to the ompi/config directory as it pertains solely to an ompi-layer component.
Forgive the mid-day configure change, but I know Shiqing is working the windows issues and don't want to cause him unnecessary redo work.
This commit was SVN r23966.
Setup the event API to support multiple bases in preparation for splitting the OMPI and ORTE events. Holding here pending shared memory resolution.
This commit was SVN r23943.
This is a fairly intrusive change, but outside of the moving of opal/event to opal/mca/event, the only changes involved (a) changing all calls to opal_event functions to reflect the new framework instead, and (b) ensuring that all opal_event_t objects are properly constructed since they are now true opal_objects.
Note: Shiqing has just returned from vacation and has not yet had a chance to complete the Windows integration. Thus, this commit almost certainly breaks Windows support on the trunk. However, I want this to have a chance to soak for as long as possible before I become less available a week from today (going to be at a class for 5 days, and thus will only be sparingly available) so we can find and fix any problems.
Biggest change is moving the libevent code from opal/event to a new opal/mca/event framework. This was done to make it much easier to update libevent in the future. New versions can be inserted as a new component and tested in parallel with the current version until validated, then we can remove the earlier version if we so choose. This is a statically built framework ala installdirs, so only one component will build at a time. There is no selection logic - the sole compiled component simply loads its function pointers into the opal_event struct.
I have gone thru the code base and converted all the libevent calls I could find. However, I cannot compile nor test every environment. It is therefore quite likely that errors remain in the system. Please keep an eye open for two things:
1. compile-time errors: these will be obvious as calls to the old functions (e.g., opal_evtimer_new) must be replaced by the new framework APIs (e.g., opal_event.evtimer_new)
2. run-time errors: these will likely show up as segfaults due to missing constructors on opal_event_t objects. It appears that it became a typical practice for people to "init" an opal_event_t by simply using memset to zero it out. This will no longer work - you must either OBJ_NEW or OBJ_CONSTRUCT an opal_event_t. I tried to catch these cases, but may have missed some. Believe me, you'll know when you hit it.
There is also the issue of the new libevent "no recursion" behavior. As I described on a recent email, we will have to discuss this and figure out what, if anything, we need to do.
This commit was SVN r23925.
#define CACHE_LINE_SIZE to 128. This name has a conflict on NetBSD,
and it seems kinda odd to have a header file that ''only'' defines a
single value. Also, we'll soon be raising hwloc to be a first-class
item, so having this file around seemed kinda weird.
Therefore, I replaced CACHE_LINE_SIZE with opal_cache_line_size, an
int (in opal/runtime/opal_init.c and opal/runtime/opal.h) on the
rationale that we can fill this in at runtime with hwloc info (trunk
and v1.5/beyond, only). The only place we ''needed'' a compile-time
CACHE_LINE_SIZE was in the BTL SM (for struct padding), so I made a
new BTL_SM_ preprocessor macro with the old CACHE_LINE_SIZE value
(128). That use isn't suitable for run-time hwloc information,
anyway.
This commit was SVN r23349.
(OMPI_ERR_* = OPAL_SOS_GET_ERR_CODE(ret)), since the return value could be a
SOS-encoded error. The OPAL_SOS_GET_ERR_CODE() takes in a SOS error and returns
back the native error code.
* Since OPAL_SUCCESS is preserved by SOS, also change all calls of the form
(OPAL_ERROR == ret) to (OPAL_SUCCESS != ret). We thus avoid having to
decode 'ret' to get the native error code.
This commit was SVN r23162.
Short version: there is a bug in OS X/Snow Leopard, but there is also
a bug in Open MPI. Fixing the bug in Open MPI is both trivial (a
1-line change) and avoids the bug in OS X. We'll file an OS X bug
report upstream with Apple, but it should no longer affect us here in
OMPI.
Fixes trac:2039.
More details:
Some background first:
1. IPv4 sockets can only accept incoming IPv4 connections. However,
IPv6 sockets can be configured to accept ''only'' incoming IPv6
connection, or ''both'' incoming IPv4 and IPv6 connections. An
IPv6 socket attribute sets which listening behavior is used.
1. IPv4 and IPv6 have different port namespaces. Hence, it is
permissable to bind a v4 socket to port X ''and'' also bind a v6
socket to that same port X on the same interface (assuming that
the v6 socket is only accepting incoming v6 connections).
Incoming v4 connections to port X on the interface should get
matched to the listening v4 socket; incoming v6 connections should
get matched to the listening v6 socket.
1. When v6 sockets accept ''both'' incoming v4 and v6 connections, it
should claim port X in both namespaces.
1. Linux's default behavior is to only allow one listening socket to
be bound to a given port (i.e., ''either'' a v6 or v4 socket to be
bound to a single port X -- not both). A v6 socket can listen for
both v4 and v6 incoming connections on that port, but still --
only one socket will be bound to that port.
1. Snow Leopard's default behavior is to share ports -- i.e., let
both a v4 and a v6 listening socket to be bound to port X
(assuming that the v6 socket is only accepting incoming v6
connections).
The TCP BTL creates a listening socket for each address family.
Hence, it creates a v4 listening socket on INADDR_ANY and a v6
listening socket on the v6 equivalent of INADDR_ANY. OMPI then
iteratively tries to find ports to listen on within the range of
[mca_btl_tcp_port_min, mca_btl_tcp_port_min + mca_btl_tcp_port_range).
On Linux, the v4 socket will be bound to port X and the v6 socket will
likely be bound to port Y (where X != Y). On Snow Leopard, the v4
socket will be bound to port X and the v6 socket may ''also'' be bound
to port X. Since the namespaces are separate, this shouldn't be a
problem.
However, Open MPI was accidentally setting the v6 listening behavior
to accept ''both'' v4 and v6 incoming connections. This is a trivial
thing to fix -- change a 0 to a 1 in the code. On Linux, this issue
didn't matter because the v4 and v6 sockets were on different ports.
So even though the v6 socket ''would'' have accepted incoming v4
connections, that never happened because OMPI would direct v4
connections to the v4 port.
But on Snow Leopard, the v4 and v6 listening ports could end up
sharing the same port number. As mentioned above, this ''shouldn't''
have been a problem, but it looks like Snow Leopard has the following
bugs:
* If a v4 socket is already bound to port X, we're pretty sure that a
v6 socket with the "accept both v4 and v6 incoming connections"
listening behavior should not be able to claim port X (because
there's already a v4 socket listening on X). However, Snow Leopard
would allow binding a v4 socket to port X, and then allow a v6
socket configured to allow incoming v4 and v6 connections to
''also'' be bound to port X.
* After binding the v6 socket to port X, Snow Leopard then lets
''another'' v4 socket ''also'' get bound to port X. Hence, there's
now '''three''' sockets all listening on port X.
This obviously led to mis-matched TCP connections, and things went
downhill from there.
That being said, Snow Leopard doesn't exhibit this behavior if v6
sockets only allow incoming v6 connections. And technically, that is
exactly the behavior we want (we want v6 sockets to only accept
incoming v6 connections). So if we just change the flag to make our
v6 listening socket us this behavior, the problem on OS X goes away.
That's what this commit does -- it changes a 0 to a 1, indicating
"only let this v6 socket allow incoming v6 connections."
That was simple, wasn't it?
This commit was SVN r22788.
The following Trac tickets were found above:
Ticket 2039 --> https://svn.open-mpi.org/trac/ompi/ticket/2039
1. The code that looks at btl_tcp_if_exclude before doing a
modex_send uses strcmp rather than strncmp. That means that
"lo0" gets sent even though "lo" is excluded.
2. The code that determines whether a particular local TCP
interface can connect to a particular remote interface doesn't
check for loopback interfaces. With this fix, users can now
enable "lo" and be assured that it will only be used for intra-
node communication.
This commit was SVN r22762.
IPv4 and IPv6) is outside the legal boundaries. This fixes trac:1869.
This commit was SVN r21612.
The following Trac tickets were found above:
Ticket 1869 --> https://svn.open-mpi.org/trac/ompi/ticket/1869
for us already.
* Slightly clarify the error message strings; now they match the new
error strings for btl_openib_ipaddr_in|exclude
This commit was SVN r21197.
subnet specifications (in addition to interface names). These
parameters now take a comma-delimited list of interfaces names and/or
a.b.c.d/x specifications (only IPv4 currently supported for subnet
specifications). For example:
mpirun --mca btl_tcp_if_include 10.10.30.0/8,eth0
This commit was SVN r21189.
OMPI_* to OPAL_*. This allows opal layer to be used more independent
from the whole of ompi.
NOTE: 9 "svn mv" operations immediately follow this commit.
This commit was SVN r21180.
- Delete unnecessary header files using
contrib/check_unnecessary_headers.sh after applying
patches, that include headers, being "lost" due to
inclusion in one of the now deleted headers...
In total 817 files are touched.
In ompi/mpi/c/ header files are moved up into the actual c-file,
where necessary (these are the only additional #include),
otherwise it is only deletions of #include (apart from the above
additions required due to notifier...)
- To get different MCAs (OpenIB, TM, ALPS), an earlier version was
successfully compiled (yesterday) on:
Linux locally using intel-11, gcc-4.3.2 and gcc-SVN + warnings enabled
Smoky cluster (x86-64 running Linux) using PGI-8.0.2 + warnings enabled
Lens cluster (x86-64 running Linux) using Pathscale-3.2 + warnings enabled
This commit was SVN r21096.
Anyway, this is blocking the move: do not include pml.h
if not really needed, aka none of the following used:
mca_pml
MCA_PML_CALL
OMPI_ANY_TAG
OMPI_ANY_SOURCE
OMPI_PROC_NULL
- Notable exceptions (deleting in one header->adding):
- ompi/mca/mtl/psm/
- ompi/mca/osc/rdma/
- ompi/mca/btl/openib/btl_openib_endpoint.c depended on
pml_base_sendreq.h
- Tested on Linux/x86-64, this time including make check
(thanks Jeff and Ralph)
This commit was SVN r20725.
anyhow -- if oob functionality is neededm then orte/mca/oob/oob.h
Nevertheless compiles fine with -Wimplicit-function-declaration
This commit was SVN r20641.
Often, orte/util/show_help.h is included, although no functionality
is required -- instead, most often opal_output.h, or
orte/mca/rml/rml_types.h
Please see orte_show_help_replacement.sh commited next.
- Local compilation (Linux/x86_64) w/ -Wimplicit-function-declaration
actually showed two *missing* #include "orte/util/show_help.h"
in orte/mca/odls/base/odls_base_default_fns.c and
in orte/tools/orte-top/orte-top.c
Manually added these.
Let's have MTT the last word.
This commit was SVN r20557.
