1
1
Граф коммитов

32 Коммитов

Автор SHA1 Сообщение Дата
Brian Barrett
27cea44a9c Fix a number of issues with the ompi_ptr_t:
* Make sure that the pval always writes to the correct portion of the
    lval.  This only matters on 32 bit big endian machines.
  * On 32 bit machines when assigning to pval, the other 4 bytes of lval
    weren't being written, which could lead to bogus data

We use macros so that there aren't casts all over the code and the pval
assignment can occur to the correct 4 bytes.  Refs trac:587

This commit was SVN r12974.

The following Trac tickets were found above:
  Ticket 587 --> https://svn.open-mpi.org/trac/ompi/ticket/587
2007-01-03 19:47:48 +00:00
George Bosilca
d8dee3a740 If the MX driver was unable to load correctly, or if the endpoint was not
created then don't try to call the MX endpoint close function.

This commit was SVN r12950.
2007-01-02 00:01:50 +00:00
George Bosilca
e223b27268 A fragment is marked completed by the PML when the peer signal the
completion of the RDMA operation associated with the fragment. The
PML will call the BML free which in turn will call the BTL free. The MX 
BTL will not release the fragment if it not tagged with 0xff.

This commit was SVN r12947.
2006-12-31 03:17:47 +00:00
George Bosilca
47601e315e Allow the MX BTL to select at runtime if the unexpected handler will
be activated or not.

This commit was SVN r12944.
2006-12-30 20:57:50 +00:00
George Bosilca
d401a65975 Minor cleanups. Don't set the fields that will never be used.
This commit was SVN r12941.
2006-12-29 07:55:17 +00:00
George Bosilca
416e5b5f6a Enable the MX extensions if and only if the mx_extensions.h header
is installed on the system.

This commit was SVN r12937.
2006-12-29 00:31:32 +00:00
George Bosilca
3eeecc3838 Add support for faster small messages. While sending a message, we check if
the data was buffered by the MX library. If it's the case then we declare
the send as completed and disable the completion event for the mx request.

This commit was SVN r12935.
2006-12-28 22:34:24 +00:00
George Bosilca
3903009b8b Add a check for the unexpected handler. If enabled, allow the zero-copy
protocol over the MX BTL. Now, we have only one matching, the one in Open
MPI.

The problem is that when the unexpected handler is triggered, not all the
message is on the host memory. In the best case we get one MX fragment (internal
MX fragment), in the worst we get NULL. The only way to fit this with the
design of the PML is to force the eager protocol at the MX internal fragment
size, and to limit the send/receive protocol at the same size. Tests show
the outcome is not far from optimal (if the pipeline depth is increased
a little bit).

Set MX_PIPELINE_LOG in order to allow MX to use internal fragments of 4K.

This commit was SVN r12930.
2006-12-28 03:35:41 +00:00
George Bosilca
ff2319dcb7 Complete the OUT protocol. Small latency improvements. Some minor cleanups.
Create some macros, reorder some functions. Make sure all fragments are
correctly released at the end.

This commit was SVN r12926.
2006-12-26 18:15:24 +00:00
George Bosilca
75a35ed7ee Implement the PUT protocol over MX. The send/receive approach give the best
performance on a 2G Myrinet card, as it look like pipelining the messages
by 1M is faster than a simple send/receive. However, when using a 10G card
the send/receive will limit the maximum bandwidth to 2.5Gbs. The reason is
the scarce bus resources that have to be shared between the Myrinet hardware
and the memcpy operation. The PUT protocol remove the memcpy, we now have a 
true zero-copy mechanism. But, there is no pipelining yet as it look like the
RDMA pipeline somehow disappeared from the OB1 PML ...

This commit was SVN r12925.
2006-12-24 22:52:46 +00:00
George Bosilca
e8bd985870 Add more output when calls to the MX library fails.
Move the connection status from theproc into the endpoint.

This commit was SVN r12924.
2006-12-24 22:34:48 +00:00
George Bosilca
dbe2798638 Allow MX to handle shared memory and self communications. By default these features
are disabled (btl_mx_shared_mem respectively btl_mx_self have to be set in order
to activate them).

This commit was SVN r12922.
2006-12-24 22:18:41 +00:00
Brian Barrett
7880353fcc Need to close every endpoint we open, or the MX progress thread doesn't die,
which can cause segfaults on shutdown.  Calling mx_finalize() isn't enough
to shutdown the thread, so must close endpoints as well.

Refs trac:513

This commit was SVN r12908.

The following Trac tickets were found above:
  Ticket 513 --> https://svn.open-mpi.org/trac/ompi/ticket/513
2006-12-21 18:13:22 +00:00
George Bosilca
80bc0c8868 Allow the MX to survive if we are unable to connect to a peer. The PML will
try to find another route.

This commit was SVN r12837.
2006-12-13 01:12:07 +00:00
George Bosilca
3d0df2cf29 Allow the MX BTL to finish the small sends quicker. Once the mx_isend is posted if
the message size is less than 4K do a check for the message completion and if any
call the callback.

This commit was SVN r12453.
2006-11-06 23:12:01 +00:00
George Bosilca
126a68dc9a Big datatype commit. Remove all unused features of the datatype engine. As the memory
allocation logic is completely done outside the data-type engine (in the PML) there is
no need for any special case inside the data-type engine. There is less arguments for
the ompi_convertor_pack and ompi_convertor_unpack as well (the last field free_after is
not required anymore as there is no memory allocated in the engine itself). This change
affect all components using datatypes. I test most of them, but it might happens that I
miss some ... If it's the case please let me know (don't shoot the pianist!!).

This commit was SVN r12331.
2006-10-26 23:11:26 +00:00
Galen Shipman
e5c594c211 More updates for the async error handler for btl's
In order to provide backwards compatability the framework versions are bumped
and the handler registeration function is at the end of the btl struct.
Testing done on sm, openib, and gm.. 

This commit was SVN r11256.
2006-08-17 22:02:01 +00:00
Galen Shipman
3b49953ce2 Add error callback to the btl interface, this allows error to be delivered to
the upperlayer assynchronously although there are some issues with this.. such
as there are multiple consumers of the btl's.. who get's the

This commit was SVN r11232.
2006-08-16 20:21:38 +00:00
George Bosilca
14b3f141db Nothing relevant !!!
This commit was SVN r10711.
2006-07-11 00:30:26 +00:00
George Bosilca
bdecdc8d41 Cleanup the MX BTL. Remove all mpool related code as there will never be a MX mpool.
This commit was SVN r9808.
2006-05-04 06:55:45 +00:00
Tim Woodall
712468dbef add diagnostic interface
This commit was SVN r9328.
2006-03-17 17:39:41 +00:00
Brian Barrett
566a050c23 Next step in the project split, mainly source code re-arranging
- move files out of toplevel include/ and etc/, moving it into the
    sub-projects
  - rather than including config headers with <project>/include, 
    have them as <project>
  - require all headers to be included with a project prefix, with
    the exception of the config headers ({opal,orte,ompi}_config.h
    mpi.h, and mpif.h)

This commit was SVN r8985.
2006-02-12 01:33:29 +00:00
George Bosilca
00c10a6372 Make the MX BTL startup scalable. When the number of processes involved in the MPI application
increase the previous connection code was broken. It can take as much as 60 seconds to connect
64 processes. Now we do not create the connections when we add the procs but only when we send
them the first message. Now it take only 1.6 seconds to setup a 64 procs MPI job over MX (doing a 2 steps barrier in order to insure that we create all the connections).

This commit was SVN r8252.
2005-11-23 23:48:56 +00:00
George Bosilca
7ad6b2b70e Add a MCA params to allow/disable the MX shared memory capabilities. Right now this param
is labeled as internal so the users will not see it but it is not read-only so we can still
play with it (that's for our internal tests). This is supposed to dissapear later after the
next (or next next) release of the MX library, but we need it now as a quick fix before the
release.

This commit was SVN r8161.
2005-11-15 20:54:45 +00:00
George Bosilca
8119c970db Improve the connection algorithm for MX. There are 2 problems here:
- first we setup the connections in the begining with all the peers
- MX does not handle well the case where several peers make connections to the same
  destination simultaneously.

So I change the order in which we connect. First we compute our rank in the array,
then in a round-robin fashion we setup connection starting with our left neighboard.

This commit was SVN r8075.
2005-11-10 01:15:49 +00:00
Jeff Squyres
42ec26e640 Update the copyright notices for IU and UTK.
This commit was SVN r7999.
2005-11-05 19:57:48 +00:00
George Bosilca
3453a6c0e9 Remove some compiler warnings about unused variables
Correctly define the 64 bits constants.
Some minor cleanups.

This commit was SVN r7606.
2005-10-04 12:29:51 +00:00
Brian Barrett
7b20370306 * pretty-print an error message if a btl component loads but can't find
any NICs to use
* Make mvapi, gm, and mx components all publish information, even if there
  are no NICs available so that modex_recv doesn't hang.  If there are no
  NICs available, don't set the reachable bit, but don't do anything
  to fail.  This unfortunately doesn't cover the hangs that will result if
  different procs load different sets of components, but it's a start

This commit was SVN r7550.
2005-09-30 04:39:44 +00:00
George Bosilca
f8d9f6121c Typo correction ...
This commit was SVN r7303.
2005-09-11 20:49:27 +00:00
George Bosilca
3078be40aa First stable version of the MX BTL (at least we pass NetPipe). The perfs are not amazing
but are not that bad either.

On a 2 procs Intel(R) Xeon(TM) CPU 3.20GHz with MYRICOM Inc. Myrinet 2000 Scalable Cluster Interconnect (rev 04) I get:

  0:       1 bytes  13096 times -->      1.10 Mbps in       6.94 usec
  1:       2 bytes  14408 times -->      2.17 Mbps in       7.02 usec
  2:       3 bytes  14243 times -->      3.24 Mbps in       7.07 usec
  3:       4 bytes   9428 times -->      4.27 Mbps in       7.15 usec
  4:       6 bytes  10493 times -->      6.26 Mbps in       7.32 usec
  5:       8 bytes   6834 times -->      8.18 Mbps in       7.47 usec
  6:      12 bytes   8371 times -->     11.89 Mbps in       7.70 usec
  7:      13 bytes   5411 times -->     12.72 Mbps in       7.80 usec
  8:      16 bytes   5919 times -->     15.35 Mbps in       7.95 usec
  9:      19 bytes   7074 times -->     17.66 Mbps in       8.21 usec
 10:      21 bytes   7696 times -->     19.00 Mbps in       8.43 usec
 11:      24 bytes   7906 times -->     20.87 Mbps in       8.77 usec
 12:      27 bytes   8073 times -->     23.05 Mbps in       8.94 usec
 13:      29 bytes   4972 times -->     24.32 Mbps in       9.10 usec
 14:      32 bytes   5307 times -->     26.29 Mbps in       9.29 usec
 15:      35 bytes   5720 times -->     33.61 Mbps in       7.95 usec
 16:      45 bytes   7191 times -->     39.50 Mbps in       8.69 usec
 17:      48 bytes   7670 times -->     41.33 Mbps in       8.86 usec
 18:      51 bytes   7759 times -->     42.80 Mbps in       9.09 usec
 19:      61 bytes   4313 times -->     47.44 Mbps in       9.81 usec
 20:      64 bytes   5012 times -->     57.61 Mbps in       8.48 usec
 21:      67 bytes   6083 times -->     59.31 Mbps in       8.62 usec
 22:      93 bytes   6234 times -->     68.08 Mbps in      10.42 usec
 23:      96 bytes   6396 times -->     80.65 Mbps in       9.08 usec
 24:      99 bytes   7455 times -->     81.56 Mbps in       9.26 usec
 25:     125 bytes   3926 times -->    112.46 Mbps in       8.48 usec
 26:     128 bytes   5848 times -->    116.87 Mbps in       8.36 usec
 27:     131 bytes   6077 times -->    119.22 Mbps in       8.38 usec
 28:     189 bytes   6192 times -->    163.79 Mbps in       8.80 usec
 29:     192 bytes   7572 times -->    168.01 Mbps in       8.72 usec
 30:     195 bytes   7705 times -->    171.13 Mbps in       8.69 usec
 31:     253 bytes   4011 times -->    210.21 Mbps in       9.18 usec
 32:     256 bytes   5423 times -->    214.55 Mbps in       9.10 usec
 33:     259 bytes   5535 times -->    217.64 Mbps in       9.08 usec
 34:     381 bytes   5613 times -->    290.55 Mbps in      10.00 usec
 35:     384 bytes   6663 times -->    296.11 Mbps in       9.89 usec
 36:     387 bytes   6764 times -->    298.74 Mbps in       9.88 usec
 37:     509 bytes   3451 times -->    353.78 Mbps in      10.98 usec
 38:     512 bytes   4546 times -->    359.36 Mbps in      10.87 usec
 39:     515 bytes   4617 times -->    361.53 Mbps in      10.87 usec
 40:     765 bytes   4645 times -->    461.41 Mbps in      12.65 usec
 41:     768 bytes   5270 times -->    468.59 Mbps in      12.50 usec
 42:     771 bytes   5341 times -->    470.16 Mbps in      12.51 usec
 43:    1021 bytes   2695 times -->    508.42 Mbps in      15.32 usec
 44:    1024 bytes   3260 times -->    514.44 Mbps in      15.19 usec
 45:    1027 bytes   3298 times -->    515.72 Mbps in      15.19 usec
 46:    1533 bytes   3307 times -->    707.12 Mbps in      16.54 usec
 47:    1536 bytes   4030 times -->    714.93 Mbps in      16.39 usec
 48:    1539 bytes   4071 times -->    714.41 Mbps in      16.44 usec
 49:    2045 bytes   2040 times -->    761.38 Mbps in      20.49 usec
 50:    2048 bytes   2438 times -->    769.78 Mbps in      20.30 usec
 51:    2051 bytes   2465 times -->    769.78 Mbps in      20.33 usec
 52:    3069 bytes   2465 times -->    923.43 Mbps in      25.36 usec
 53:    3072 bytes   2629 times -->    928.48 Mbps in      25.24 usec
 54:    3075 bytes   2642 times -->    929.07 Mbps in      25.25 usec
 55:    4093 bytes   1323 times -->   1012.38 Mbps in      30.85 usec
 56:    4096 bytes   1620 times -->   1016.69 Mbps in      30.74 usec
 57:    4099 bytes   1627 times -->   1015.16 Mbps in      30.81 usec
 58:    6141 bytes   1625 times -->   1171.82 Mbps in      39.98 usec
 59:    6144 bytes   1667 times -->   1173.85 Mbps in      39.93 usec
 60:    6147 bytes   1669 times -->   1174.44 Mbps in      39.93 usec
 61:    8189 bytes    835 times -->   1232.43 Mbps in      50.69 usec
 62:    8192 bytes    986 times -->   1234.87 Mbps in      50.61 usec
 63:    8195 bytes    988 times -->   1234.85 Mbps in      50.63 usec
 64:   12285 bytes    988 times -->   1360.73 Mbps in      68.88 usec
 65:   12288 bytes    967 times -->   1364.20 Mbps in      68.72 usec
 66:   12291 bytes    970 times -->   1364.56 Mbps in      68.72 usec
 67:   16381 bytes    485 times -->   1385.48 Mbps in      90.21 usec
 68:   16384 bytes    554 times -->   1388.76 Mbps in      90.01 usec
 69:   16387 bytes    555 times -->   1388.41 Mbps in      90.05 usec
 70:   24573 bytes    555 times -->   1499.72 Mbps in     125.01 usec
 71:   24576 bytes    533 times -->   1499.36 Mbps in     125.05 usec
 72:   24579 bytes    533 times -->   1500.44 Mbps in     124.98 usec
 73:   32765 bytes    266 times -->   1499.31 Mbps in     166.73 usec
 74:   32768 bytes    299 times -->   1497.10 Mbps in     166.99 usec
 75:   32771 bytes    299 times -->   1495.29 Mbps in     167.21 usec
 76:   49149 bytes    299 times -->   1528.78 Mbps in     245.28 usec
 77:   49152 bytes    271 times -->   1527.97 Mbps in     245.42 usec
 78:   49155 bytes    271 times -->   1529.35 Mbps in     245.22 usec
 79:   65533 bytes    135 times -->   1586.19 Mbps in     315.21 usec
 80:   65536 bytes    158 times -->   1591.11 Mbps in     314.25 usec
 81:   65539 bytes    159 times -->   1586.50 Mbps in     315.17 usec
 82:   98301 bytes    158 times -->   1668.05 Mbps in     449.61 usec
 83:   98304 bytes    148 times -->   1667.40 Mbps in     449.80 usec
 84:   98307 bytes    148 times -->   1667.29 Mbps in     449.84 usec
 85:  131069 bytes     74 times -->   1709.11 Mbps in     585.09 usec
 86:  131072 bytes     85 times -->   1711.09 Mbps in     584.42 usec
 87:  131075 bytes     85 times -->   1710.92 Mbps in     584.49 usec
 88:  196605 bytes     85 times -->   1727.93 Mbps in     868.08 usec
 89:  196608 bytes     76 times -->   1726.28 Mbps in     868.92 usec
 90:  196611 bytes     76 times -->   1727.06 Mbps in     868.54 usec
 91:  262141 bytes     38 times -->   1757.65 Mbps in    1137.87 usec
 92:  262144 bytes     43 times -->   1758.69 Mbps in    1137.21 usec
 93:  262147 bytes     43 times -->   1759.38 Mbps in    1136.78 usec
 94:  393213 bytes     43 times -->   1801.51 Mbps in    1665.25 usec
 95:  393216 bytes     40 times -->   1803.26 Mbps in    1663.65 usec
 96:  393219 bytes     40 times -->   1800.73 Mbps in    1666.00 usec
 97:  524285 bytes     20 times -->   1805.33 Mbps in    2215.65 usec
 98:  524288 bytes     22 times -->   1806.80 Mbps in    2213.86 usec
 99:  524291 bytes     22 times -->   1805.77 Mbps in    2215.14 usec
100:  786429 bytes     22 times -->   1827.24 Mbps in    3283.64 usec
101:  786432 bytes     20 times -->   1827.03 Mbps in    3284.03 usec
102:  786435 bytes     20 times -->   1827.20 Mbps in    3283.73 usec
103: 1048573 bytes     10 times -->   1840.05 Mbps in    4347.71 usec
104: 1048576 bytes     11 times -->   1839.68 Mbps in    4348.58 usec
105: 1048579 bytes     11 times -->   1840.13 Mbps in    4347.54 usec
106: 1572861 bytes     11 times -->   1853.99 Mbps in    6472.50 usec
107: 1572864 bytes     10 times -->   1854.11 Mbps in    6472.10 usec
108: 1572867 bytes     10 times -->   1854.12 Mbps in    6472.10 usec
109: 2097149 bytes      5 times -->   1861.41 Mbps in    8595.61 usec
110: 2097152 bytes      5 times -->   1861.25 Mbps in    8596.40 usec
111: 2097155 bytes      5 times -->   1860.99 Mbps in    8597.59 usec
112: 3145725 bytes      5 times -->   1868.34 Mbps in   12845.59 usec
113: 3145728 bytes      5 times -->   1868.30 Mbps in   12845.90 usec
114: 3145731 bytes      5 times -->   1868.59 Mbps in   12843.89 usec
115: 4194301 bytes      3 times -->   1872.16 Mbps in   17092.51 usec
116: 4194304 bytes      3 times -->   1872.31 Mbps in   17091.19 usec
117: 4194307 bytes      3 times -->   1872.13 Mbps in   17092.82 usec
118: 6291453 bytes      3 times -->   1875.88 Mbps in   25588.00 usec
119: 6291456 bytes      3 times -->   1875.98 Mbps in   25586.68 usec
120: 6291459 bytes      3 times -->   1875.93 Mbps in   25587.36 usec
121: 8388605 bytes      3 times -->   1877.79 Mbps in   34082.69 usec
122: 8388608 bytes      3 times -->   1877.72 Mbps in   34083.84 usec
123: 8388611 bytes      3 times -->   1877.66 Mbps in   34085.00 usec

This commit was SVN r7180.
2005-09-04 22:08:13 +00:00
George Bosilca
c8bc529df4 The second cut of MX ... still not working yet
This commit was SVN r6666.
2005-07-28 19:53:27 +00:00
George Bosilca
e1b3758fa5 The first cut for he MX BTL.
This commit was SVN r6621.
2005-07-27 19:46:36 +00:00