1
1
Граф коммитов

26 Коммитов

Автор SHA1 Сообщение Дата
Pavel Shamis
f0c478e7e0 XRC - replacing the new old API with new one.
This commit was SVN r17369.
2008-02-04 14:03:38 +00:00
Jeff Squyres
2227d5ec4a Add configure check for struct ibv_device.transport type, which was added in OFED v1.2. Still need to fix up oob and rdma_cm cpc's to do something better with this information...
This commit was SVN r17198.
2008-01-24 12:14:21 +00:00
Jon Mason
a0d4122606 The new cpc selection framework is now in place. The patch below allows
for dynamic selection of cpc methods based on what is available.  It
also allows for inclusion/exclusions of methods.  It even futher allows
for modifying the priorities of certain cpc methods to better determine
the optimal cpc method.

This patch also contains XRC compile time disablement (per Jeff's
patch).

At a high level, the cpc selections works by walking through each cpc
and allowing it to test to see if it is permissable to run on this
mpirun.  It returns a priority if it is permissable or a -1 if not.  All
of the cpc names and priorities are rolled into a string.  This string
is then encapsulated in a message and passed around all the ompi
processes.  Once received and unpacked, the list received is compared
to a local copy of the list.  The connection method is chosen by
comparing the lists passed around to all nodes via modex with the list
generated locally.  Any non-negative number is a potentially valid
connection method.  The method below of determining the optimal
connection method is to take the cross-section of the two lists.  The
highest single value (and the other side being non-negative) is selected
as the cpc method.

svn merge -r 16948:17128 https://svn.open-mpi.org/svn/ompi/tmp-public/openib-cpc/ .

This commit was SVN r17138.
2008-01-14 23:22:03 +00:00
Gleb Natapov
bd47da4699 Initial XRC support by Mellanox.
This commit was SVN r16787.
2007-11-28 07:18:59 +00:00
Jeff Squyres
8ace07efed This commit brings in two major things:
1. Galen's fine-grain control of queue pair resources in the openib
   BTL.
1. Pasha's new implementation of asychronous HCA event handling.

Pasha's new implementation doesn't take much explanation, but the new
"multifrag" stuff does.  

Note that "svn merge" was not used to bring this new code from the
/tmp/ib_multifrag branch -- something Bad happened in the periodic
trunk pulls on that branch making an actual merge back to the trunk
effectively impossible (i.e., lots and lots of arbitrary conflicts and
artifical changes).  :-(

== Fine-grain control of queue pair resources ==

Galen's fine-grain control of queue pair resources to the OpenIB BTL
(thanks to Gleb for fixing broken code and providing additional
functionality, Pasha for finding broken code, and Jeff for doing all
the svn work and regression testing).

Prior to this commit, the OpenIB BTL created two queue pairs: one for
eager size fragments and one for max send size fragments.  When the
use of the shared receive queue (SRQ) was specified (via "-mca
btl_openib_use_srq 1"), these QPs would use a shared receive queue for
receive buffers instead of the default per-peer (PP) receive queues
and buffers.  One consequence of this design is that receive buffer
utilization (the size of the data received as a percentage of the
receive buffer used for the data) was quite poor for a number of
applications.

The new design allows multiple QPs to be specified at runtime.  Each
QP can be setup to use PP or SRQ receive buffers as well as giving
fine-grained control over receive buffer size, number of receive
buffers to post, when to replenish the receive queue (low water mark)
and for SRQ QPs, the number of outstanding sends can also be
specified.  The following is an example of the syntax to describe QPs
to the OpenIB BTL using the new MCA parameter btl_openib_receive_queues:

{{{
-mca btl_openib_receive_queues \
     "P,128,16,4;S,1024,256,128,32;S,4096,256,128,32;S,65536,256,128,32"
}}}

Each QP description is delimited by ";" (semicolon) with individual
fields of the QP description delimited by "," (comma).  The above
example therefore describes 4 QPs.

The first QP is:

    P,128,16,4

Meaning: per-peer receive buffer QPs are indicated by a starting field
of "P"; the first QP (shown above) is therefore a per-peer based QP.
The second field indicates the size of the receive buffer in bytes
(128 bytes).  The third field indicates the number of receive buffers
to allocate to the QP (16).  The fourth field indicates the low
watermark for receive buffers at which time the BTL will repost
receive buffers to the QP (4).

The second QP is:

    S,1024,256,128,32

Shared receive queue based QPs are indicated by a starting field of
"S"; the second QP (shown above) is therefore a shared receive queue
based QP.  The second, third and fourth fields are the same as in the
per-peer based QP.  The fifth field is the number of outstanding sends
that are allowed at a given time on the QP (32).  This provides a
"good enough" mechanism of flow control for some regular communication
patterns.

QPs MUST be specified in ascending receive buffer size order.  This
requirement may be removed prior to 1.3 release.

This commit was SVN r15474.
2007-07-18 01:15:59 +00:00
Jeff Squyres
36679de8d8 Fixes trac:1045.
libsysfs headers are required for libibverbs v1.0 (i.e., OFED 1.0 and
OFED 1.1), meaning that <infiniband/verbs.h> would #include
<sysfs/libsysfs.h>.  Hence, if the libsysfs headers did not exist on a
system, including <verbs.h> would fail.  

With older versions of Autoconf, we would simply test for the
''presence'' of the <infinband/verbs.h> and not actually try to
''use'' it.  This could leave OMPI in a weird situation on systems
that did not have the sysfs headers installed: configure would
complete successfully, but the build of the openib btl would fail.
Some users complained, thinking that there was a real compile error in
the OMPI code base.

Hence, we decided that it would be better to AC_CHECK_HEADER for the
sysfs header files in configure.  If the sysfs header files were not
found, configure would abort.  Users generally understand when
configure aborts, and know how to read the output and fix the
underlying problem; it was ''much'' more obvious than having the OMPI
build fail for nebulous reasons much later.

Note that we also checked for / added -lsysfs, but that wasn't
necessary because libibverbs already run-time linked to it (i.e.,
libibverbs couldn't have been installed if the sysfs libraries weren't
installed).

However, there are now two reasons why the check for sysfs's header
files is no longer necessary:

 * Newer versions of Autoconf are now used for OMPI tarballs that
   check for both the presence '''and''' usability of header files.
   Hence, AC_CHECK_HEADER for <infiniband/verbs.h> will actually try
   to ''use'' it, so if the sysfs header files are not installed,
   AC_CHECK_HEADER will (rightfully) fail.
 * libibverbs v1.1 (i.e., OFED 1.2 and beyond) does not require
   libsysfs at all (headers or libraries).  

When checking for the sysfs header files, OMPI's configure ''forces''
you to have sysfs installed, even though it may not be needed (e.g.,
libibverbs v1.1 and beyond).  Clearly, this is not good (especially
since the sysfs software package is now deprecated, and some Linux
distros no longer install it by default).

So this commit simply removes the check for the sysfs header files and
libraries, allowing OMPI to be build on systems with libibverbs >=1.1 that
do not have sysfs installed.

For systems with libibverbs 1.0, if they do not have the sysfs headers
installed, we'll still fail AC_CHECK_HEADER and therefore still fail
configure properly.  I expanded the warning message to say that if
libibverbs 1.0 is being used, check to ensure that sysfs is installed,
yadda yadda yadda.

This commit was SVN r14971.

The following Trac tickets were found above:
  Ticket 1045 --> https://svn.open-mpi.org/trac/ompi/ticket/1045
2007-06-08 23:34:05 +00:00
Brian Barrett
a446af5b6b * Remove unneeded SRQ test -- we no longer support OFED builds that don't
have the SRQ interface.
  * Instead of setting AC_DEFINEs per MCA component, set per test.  THe
    answers can never be difference, and this will speed sed just a teeny
    bit

This commit was SVN r14856.
2007-06-05 01:49:26 +00:00
Pavel Shamis
cd87b05711 Added check for IBV_EVENT_CLIENT_REREGISTER async
event that was not exists in old openib gen2 versions
(Ticket #1025)

This commit was SVN r14658.
2007-05-15 13:53:49 +00:00
Brian Barrett
e63346a633 Clean up a couple of issues with some of the configure tests:
* Remove duplicate calls to ompi_check_package by poking at the
    internals just a bit.  Possibly should make those officially
    exposed, but whatever.
  * Don't build OpenIB with PTMalloc2 and no thread support, as
    this will always lead to badness.
  * minor formatting cleanups

This commit was SVN r14552.
2007-05-01 04:40:31 +00:00
Jeff Squyres
0ba47105ed Merge the /tmp/jms-installdirs-trunk branch into the trunk. This
finally brings in functionality that is already on the 1.2 branch, and
was developed and tested in the v1.2ofed branch (and other places).

Short version of new features:

 * Support for ibv_fork_init() 
 * Automatically fill in the openib BTL bandwidth value by 
   querying the HCA port 
 * Installdirs functionality 
 * Fixes to always use -I in the Fortran wrapper compilers (#924) 
 * Gleb's mpool updates 
 * Remove some kruft in btl/openib/configure.m4, therefore 
   fixing the harmless warnings noted in #665 
 * Bunches of updates to the Linux RPM spec file 

I.e., effectively the same thing that r14411 brought to the v1.2
branch.

Also effectively brought in r14432 and r14433 (some fixes on top of
the original r14411 commit to v1.2).  Still need to bring in the moral
equivalent of r14445 after this commit (fixes to installdirs).

This commit was SVN r14449.

The following SVN revision numbers were found above:
  r14411 --> open-mpi/ompi@83b31314ae
  r14432 --> open-mpi/ompi@a48f160595
  r14433 --> open-mpi/ompi@68f346d2bc
  r14445 --> open-mpi/ompi@13d366b827
2007-04-21 00:15:05 +00:00
Pavel Shamis
edeab0e912 Adding Mellanox Technologies copyright to files touched by Mellanox.
This commit was SVN r13669.
2007-02-15 18:03:20 +00:00
Pavel Shamis
566667ac61 Adding progress thread support to OpenIB BTL.
Reviewed by Gleb.

This commit was SVN r12411.
2006-11-02 16:15:21 +00:00
Galen Shipman
c9e0eda190 Initialize the completion queue to a reasonable size based on maximum number
of send/receives outstanding.

Use ibv_cq_resize if available after initial creation of completion queue if
cq_size is too small (based on number of peers). 

This commit was SVN r11053.
2006-07-30 00:58:40 +00:00
Brian Barrett
56a86c89b1 * Fix for bug #152. Make --with-FOO-libdir=DIR only check DIR, and not the
DIR/lib and DIR/lib64, which really never made sense.

This commit was SVN r10535.
2006-06-28 03:33:30 +00:00
Jeff Squyres
d4b5c98247 Add check for <sysfs/libsysfs.h> since it's needed by OpenIB.
This commit was SVN r10213.
2006-06-05 22:02:40 +00:00
Brian Barrett
5163f2b296 Fix for bug #36. The MX, MVAPI, and OpenIB components don't have
support for progress threads, so we shouldn't build them or try to use
them when support for progress threads has been requested.  The TCP, GM,
SELF, and SM BTLs should have progress thread support, so they aren't
disabled.  The Portals BTL isn't compiled on platforms with threads,
so it doens't need to be updated.

This commit was SVN r10156.
2006-06-01 01:30:16 +00:00
Jeff Squyres
74d759dc47 Patch submitted by Roland Dreier to look for a change in the OpenIB
API.

This commit was SVN r8687.
2006-01-13 23:20:49 +00:00
Brian Barrett
08799e4a48 * Make the format of the --with-BTL options the same for all the BTLs
* Make sure --without-BTL works for all BTLs
* Fix copy-n-paste error in aix timer configrue help string

This should go to the v1.0 branch

This commit was SVN r8554.
2005-12-20 00:47:36 +00:00
Galen Shipman
ff1a820384 set the lib path correctly, build system doesn't like white space..
This commit was SVN r8291.
2005-11-28 21:08:39 +00:00
Galen Shipman
209bc9ca93 fix..
This commit was SVN r8288.
2005-11-28 20:44:53 +00:00
Jeff Squyres
42ec26e640 Update the copyright notices for IU and UTK.
This commit was SVN r7999.
2005-11-05 19:57:48 +00:00
Brian Barrett
57781484ef * check for shared receive queue support, and set OMPI_MCA_BTL_OPENIB_HAVE_SRQ
if we have said support

This commit was SVN r7575.
2005-10-01 18:44:31 +00:00
Brian Barrett
997644af31 * There are now two forms of ibv_create_cq, one with 3 params and one with 5.
Try to detect which form this version of Open IB uses, defaulting to the 5
  version if we can't figure it out (the new version has 5 params)
* Only add -lcm if it exists on the system - some versions of Open IB
  apparently don't need it.

This commit was SVN r7542.
2005-09-29 13:35:57 +00:00
Jeff Squyres
5702c8f29b Fix help messages and docs about --with arguments
This commit was SVN r7183.
2005-09-05 18:51:34 +00:00
Brian Barrett
44ae604451 * Don't let -lsysfs propogate outside of the OpenIB components. Even if
OpenIB wasn't found, it was getting put in LIBS and causing issues on
  some platforms...

This commit was SVN r6718.
2005-08-02 20:25:42 +00:00
Galen Shipman
f3843bee55 Updated openib btl and mpool to use configure.m4
removed ompi_ignore files from openib btl and mpool. 

This commit was SVN r6612.
2005-07-27 03:38:25 +00:00