Short version: there is a bug in OS X/Snow Leopard, but there is also
a bug in Open MPI. Fixing the bug in Open MPI is both trivial (a
1-line change) and avoids the bug in OS X. We'll file an OS X bug
report upstream with Apple, but it should no longer affect us here in
OMPI.
Fixes trac:2039.
More details:
Some background first:
1. IPv4 sockets can only accept incoming IPv4 connections. However,
IPv6 sockets can be configured to accept ''only'' incoming IPv6
connection, or ''both'' incoming IPv4 and IPv6 connections. An
IPv6 socket attribute sets which listening behavior is used.
1. IPv4 and IPv6 have different port namespaces. Hence, it is
permissable to bind a v4 socket to port X ''and'' also bind a v6
socket to that same port X on the same interface (assuming that
the v6 socket is only accepting incoming v6 connections).
Incoming v4 connections to port X on the interface should get
matched to the listening v4 socket; incoming v6 connections should
get matched to the listening v6 socket.
1. When v6 sockets accept ''both'' incoming v4 and v6 connections, it
should claim port X in both namespaces.
1. Linux's default behavior is to only allow one listening socket to
be bound to a given port (i.e., ''either'' a v6 or v4 socket to be
bound to a single port X -- not both). A v6 socket can listen for
both v4 and v6 incoming connections on that port, but still --
only one socket will be bound to that port.
1. Snow Leopard's default behavior is to share ports -- i.e., let
both a v4 and a v6 listening socket to be bound to port X
(assuming that the v6 socket is only accepting incoming v6
connections).
The TCP BTL creates a listening socket for each address family.
Hence, it creates a v4 listening socket on INADDR_ANY and a v6
listening socket on the v6 equivalent of INADDR_ANY. OMPI then
iteratively tries to find ports to listen on within the range of
[mca_btl_tcp_port_min, mca_btl_tcp_port_min + mca_btl_tcp_port_range).
On Linux, the v4 socket will be bound to port X and the v6 socket will
likely be bound to port Y (where X != Y). On Snow Leopard, the v4
socket will be bound to port X and the v6 socket may ''also'' be bound
to port X. Since the namespaces are separate, this shouldn't be a
problem.
However, Open MPI was accidentally setting the v6 listening behavior
to accept ''both'' v4 and v6 incoming connections. This is a trivial
thing to fix -- change a 0 to a 1 in the code. On Linux, this issue
didn't matter because the v4 and v6 sockets were on different ports.
So even though the v6 socket ''would'' have accepted incoming v4
connections, that never happened because OMPI would direct v4
connections to the v4 port.
But on Snow Leopard, the v4 and v6 listening ports could end up
sharing the same port number. As mentioned above, this ''shouldn't''
have been a problem, but it looks like Snow Leopard has the following
bugs:
* If a v4 socket is already bound to port X, we're pretty sure that a
v6 socket with the "accept both v4 and v6 incoming connections"
listening behavior should not be able to claim port X (because
there's already a v4 socket listening on X). However, Snow Leopard
would allow binding a v4 socket to port X, and then allow a v6
socket configured to allow incoming v4 and v6 connections to
''also'' be bound to port X.
* After binding the v6 socket to port X, Snow Leopard then lets
''another'' v4 socket ''also'' get bound to port X. Hence, there's
now '''three''' sockets all listening on port X.
This obviously led to mis-matched TCP connections, and things went
downhill from there.
That being said, Snow Leopard doesn't exhibit this behavior if v6
sockets only allow incoming v6 connections. And technically, that is
exactly the behavior we want (we want v6 sockets to only accept
incoming v6 connections). So if we just change the flag to make our
v6 listening socket us this behavior, the problem on OS X goes away.
That's what this commit does -- it changes a 0 to a 1, indicating
"only let this v6 socket allow incoming v6 connections."
That was simple, wasn't it?
This commit was SVN r22788.
The following Trac tickets were found above:
Ticket 2039 --> https://svn.open-mpi.org/trac/ompi/ticket/2039
Cleanup the kill_procs command by removing a no-longer-used param. We update the process state when the proc actually exits.
This commit was SVN r22783.
1. The code that looks at btl_tcp_if_exclude before doing a
modex_send uses strcmp rather than strncmp. That means that
"lo0" gets sent even though "lo" is excluded.
2. The code that determines whether a particular local TCP
interface can connect to a particular remote interface doesn't
check for loopback interfaces. With this fix, users can now
enable "lo" and be assured that it will only be used for intra-
node communication.
This commit was SVN r22762.
btl_openib_ip.*. The routines in these files are not specific to
iwarp -- they are specific to IP interfaces used with IBV devices
(even IB or IBoE/RoCEE/whatever devices).
This commit was SVN r22718.
issues with iwarp.c. These fixes are needed for IBoE / ROCEE /
whateveritscalledtoday. I added a few minor changes to his base
patch.
This commit was SVN r22717.
libtool.m4's are not newer than aclocal.m4. They "usually weren't",
but if you had a slow filesystem, it could be possible that libtool.m4
would be newer than aclocal.m4, and Bad Things would happen during
"make" (i.e., running configure again).
This commit was SVN r22715.
Modify the orte configure options to specify --enable-multicast such that it directs components to build or not instead of littering the code base with #if's. Remove those #if's where they used to occur.
Add a new grpcomm "mcast" module to support multicast operations. Still some work required to properly perform daemon collectives for comm_spawn operations. New module only builds when --enable-multicast is provided, and when specifically selected.
This commit was SVN r22709.
bug: libmpi_f90 had libmpi.la in its LIBADD instead of libmpi_f77.la.
Fixes trac:2244.
This commit was SVN r22704.
The following Trac tickets were found above:
Ticket 2244 --> https://svn.open-mpi.org/trac/ompi/ticket/2244
