1
1

831 Коммитов

Автор SHA1 Сообщение Дата
Gleb Natapov
8078ae5977 Optimize sm communication. Pass message type (MCA_BTL_SM_FRAG_ACK/
MCA_BTL_SM_FRAG_SEND) and status success/fail in low bits of pointers we
are passing through circular buffer. The rank that receives ACK doesn't need
to look into data it received and this is a big win since this data is not in
the cache of the rank's CPU. (Note that we can use low bits of pointers because
free_list always return pointers aligned at least to cache line size).

This commit was SVN r13922.
2007-03-05 14:24:09 +00:00
Gleb Natapov
90fb58de4f When frags are allocated from mpool by free_list the frag structure is also
allocated from mpool memory (which is registered memory for RDMA transports)
This is not a problem for a small jobs, but for a big number of ranks an
amount of waisted memory is big.

This commit was SVN r13921.
2007-03-05 14:17:50 +00:00
Rich Graham
e932d9a695 macro variable has same name as one of the parameters passed to the
macro.
Typo - most likely cut and paste error.

This commit was SVN r13918.
2007-03-04 23:31:07 +00:00
Josh Hursey
0404444dbe * Added 2 new MCA parameters
- mca_base_param_file_prefix
     (Default: NULL)
     This is the fullname of the "-am" mpirun option. Used to specify a ':'
     separated list of AMCA parameter set files.
  - mca_base_param_file_path
     (Default: $SYSCONFDIR/amca-param-sets/:$CWD)
     The path to search for AMCA files with relative paths. A warning will be
     printed if the AMCA file cannot be found.

* Added a new function "mca_base_param_recache_files" the re-reads the file
configurations. This is used internally to help bootstrap the MCA system.

* Added a new orterun/mpirun command line option '-am' that aliases for the
mca_base_param_file_prefix MCA parameter

* Exposed the opal_path_access function as it is generally useful in other
places in the code.

* New function "opal_cmd_line_make_opt_mca" which will allow you to append a
new command line option with MCA parameter identifiers to set at the same
time. Previously this could only be done at command line declaration time.

* Added a new directory under the $pkgdatadir named "amca-param-sets" where all
the 'shipped with' Open MPI AMCA parameter sets are placed. This is the first
place to search for AMCA sets with relative paths.

* An example.conf AMCA parameter set file is located in
contrib/amca-param-sets/.

* Jeff Squyres contributed an OpenIB AMCA set for benchmarking.

Note: You will need to autogen with this commit as it adds a configure param.
  Sorry :(

This commit was SVN r13867.
2007-03-01 13:39:20 +00:00
Gleb Natapov
2b6cbd6299 Separate frag lists for RDMA descriptors to two, one for src descriptors
and another for dst descriptors. This provide partial solution to OB1 protocol
deadlock problem. We can limit number of RDMA descriptors (by setting
btl_openib_free_list_max to something different from -1) and if we will be
lucky to hit this limit before we fail to register more memory the protocol
will not deadlock. When we had only one list for src/dst descriptors we
deadlocked when we reached max limit for the list.

This commit was SVN r13844.
2007-02-28 13:43:38 +00:00
Sven Stork
d8a369936e - Fix more symbols that should be exported.
This commit was SVN r13824.
2007-02-27 15:17:17 +00:00
Josh Hursey
c573171b7d Mostly a cleanup commit.
- Implement the BML/r2 finialize funciton
- Cleanup the btl close routine
- Wire up a pml_base_verbose MCA parameter so you can actually watch the PML selection logic if you really want to.
- Fix a potental segfault in the selection logic.
  ompi_pointer_array_get_item() may return NULL, so we have to check for it

This commit was SVN r13734.

The following SVN revision numbers were found above:
  r2 --> open-mpi/ompi@58fdc18855
2007-02-21 16:18:43 +00:00
Jeff Squyres
f820e44112 Remove a gcc-ism from the code (defining an anonymous union in the
middle of a struct).  Now we properly define and name the union
outside the struct and simply create an instance of it inside the
struct. 

This commit was SVN r13709.
2007-02-19 18:21:57 +00:00
George Bosilca
020b8ade70 A slightly better fix for the data mismatch compiler complaints.
This commit was SVN r13695.
2007-02-17 05:23:57 +00:00
George Bosilca
04138c23af No more warnings.
This commit was SVN r13683.
2007-02-16 16:25:58 +00:00
Pavel Shamis
edeab0e912 Adding Mellanox Technologies copyright to files touched by Mellanox.
This commit was SVN r13669.
2007-02-15 18:03:20 +00:00
Gleb Natapov
4d4b0a022a Add error callback to sm BTL. Call it when allocation of the initial circular
buffer fails. If cb is already allocated, but it is full and allocation of
additional cb fails, we spin waiting for receiver to free space in existing
cb.

This commit was SVN r13635.
2007-02-13 12:01:36 +00:00
Galen Shipman
f98a442c82 Fix a problem in the selection logic for MX. Basically we need to be able to
open MTL MX and BTL MX and initialize them at the same time. The problem is
that both call mx_init and mx_finalize, solution is to add an external entity
that does the init and finalize (based on ref counting).

This commit was SVN r13576.
2007-02-09 03:19:38 +00:00
Jeff Squyres
c91fcd7fbd Fix a bunch of minor typos submitted by Bernhard Fischer.
This commit was SVN r13505.
2007-02-06 12:00:30 +00:00
Galen Shipman
a94101fa62 mostly another hack around for PML selection, allows CM be select itself if an
MTL is available, if not OB1 is used. Still prevents DR and OB1 from stomping
on each other though. 

This commit was SVN r13481.
2007-02-03 02:01:18 +00:00
George Bosilca
0ff2115964 Other warnings are now silenced.
This commit was SVN r13462.
2007-02-02 06:47:35 +00:00
George Bosilca
56ffbfc5ff Get rid of the warnings in the Open IB BTL.
This commit was SVN r13424.
2007-02-01 19:07:04 +00:00
George Bosilca
b611e6d7dc Less warnings.
This commit was SVN r13419.
2007-02-01 17:51:43 +00:00
George Bosilca
6ef3917741 Allow the user to specify the bandwidth and latency for the MX device.
This commit was SVN r13418.
2007-02-01 17:51:00 +00:00
Brian Barrett
58b325b03f Two changes to improve the sm situation with spawn:
* have the mpool size be based on MCW, not num procs
    in other jobs we know about.  Solves the problem of
    the spawned job having a much bigger than needed
    sm file
  * Can't assume that "me" is in the list of procs
    passed to addprocs, so need to use slightly different
    logic and not go through all of add procs unless
    there's a proc in my job that isn't me.

This seems to greatly improve the situation, although
there still seems to be more of a slowdown through
MPI_INIT for the children (if there are more than one
child) than MPI_INIT for the parent if there are 'n'
children compared to 'n' parents.  Hopefully that
made sense ;)

This commit was SVN r13417.
2007-02-01 17:18:35 +00:00
Brian Barrett
ee753694e0 Print out the memlock limit when we can't allocate memory
This commit was SVN r13372.
2007-01-30 21:22:56 +00:00
Jeff Squyres
86f8c66a27 Turns out that the leave_pinned stuff isn't used in these BTLs at
all.  So just remove it.

This commit was SVN r13360.
2007-01-30 15:39:49 +00:00
Jeff Squyres
c9f072b84f Strike down a few more stray places that were registering
mpi_leave_pinned and replace them with the one central global
variable.

This commit was SVN r13349.
2007-01-29 20:24:31 +00:00
Jeff Squyres
7b6ed64c7b Add in the hostname to the BTL_* output macros so that you can tell on
which node an event occurred.

This commit was SVN r13302.
2007-01-25 14:02:54 +00:00
Jeff Squyres
6fea000e5f Oops -- get the right function name (copy-n-paste error).
This commit was SVN r13290.
2007-01-24 22:31:13 +00:00
Jeff Squyres
6b69ea664d Make a much, much better error message for a not-uncommon failure
scenario (user/sysadmin forgot to set the memlock limits high
enough).

This commit was SVN r13289.
2007-01-24 22:25:40 +00:00
Jeff Squyres
c9fe68c406 Better patch from Gleb to do the per-port (endpoint) specification of
whether to use eager RDMA or not

This commit was SVN r13262.
2007-01-23 22:40:59 +00:00
Jeff Squyres
3389a523e9 Arrgh. That printf should not have been in there!
This commit was SVN r13243.
2007-01-22 18:52:49 +00:00
Jeff Squyres
a24f3c0886 Move the "use eager RDMA" flag to the individual openib BTL modules,
not the component.  This potentially allows for a mix of HCAs that
support eager RDMA and those who do not on a port-by-port basis.

This commit was SVN r13242.
2007-01-22 18:49:32 +00:00
Jeff Squyres
91b855c2f4 Minor fixes in the help messages
* If the text to cite where the problem occurred is "\n", prettyprint
   somethign a little nicer so that it's clear that we're talking
   about the end of line
 * Add a missing help message ("ini file:unknown field"), and display
   it a little better (i.e., show the erroneous field, not a
   misleading "end of line" marker)
 * It's "OpenIB", not "Open IB"

This commit was SVN r13241.
2007-01-22 18:45:43 +00:00
Jeff Squyres
e934272f3e Commit data supplied by Christian Bell at QLogic for their vendor and
part ID's.

This commit was SVN r13216.
2007-01-19 19:46:29 +00:00
Jeff Squyres
754042f1fc Fix a compiler warning.
This commit was SVN r13134.
2007-01-16 23:03:17 +00:00
Brian Barrett
e93eaa0790 Remote pointers are always in .lval, not .pval, so need to read the .lval
and convert it to a pointer when finding the destination addr.

Refs trac:587

This commit was SVN r13116.

The following Trac tickets were found above:
  Ticket 587 --> https://svn.open-mpi.org/trac/ompi/ticket/587
2007-01-14 20:34:41 +00:00
Jeff Squyres
d5404f21a3 Make the trunk openib btl compile again.
This commit was SVN r13110.
2007-01-13 14:22:42 +00:00
Galen Shipman
4a6ad30440 remove unused macro calls..
This commit was SVN r13107.
2007-01-12 23:17:17 +00:00
Galen Shipman
2097d174f6 heterogeneous fixes to the OpenIB BTL. This includes work by nysal, brian and
I. 

This commit was SVN r13106.
2007-01-12 23:14:45 +00:00
Galen Shipman
df099a4731 call it what it is...
we are looking at subnet_id's and we are counting active ports per subnet. 
move subnet count out of procs loop,, no need to do it there... 

This commit was SVN r13105.
2007-01-12 22:42:20 +00:00
Brian Barrett
075161afa9 Enable MX wireup in heterogeneous situations.
Refs trac:587

This commit was SVN r13095.

The following Trac tickets were found above:
  Ticket 587 --> https://svn.open-mpi.org/trac/ompi/ticket/587
2007-01-12 15:44:58 +00:00
Donald Kerr
ed097d17c1 fix for bug #749, though I can not confirm without a linux compiler
This commit was SVN r13090.
2007-01-11 22:25:13 +00:00
Donald Kerr
80f2cbb498 add udapl rdma capabilities into the udapl btl
This commit was SVN r13082.
2007-01-11 15:22:08 +00:00
Jeff Squyres
e5205657cf A much better fix for #739. No configure test -- just do a simple
memcpy() instead of assigning the struct's by value.

Fixes trac:739.

This commit was SVN r13081.

The following Trac tickets were found above:
  Ticket 739 --> https://svn.open-mpi.org/trac/ompi/ticket/739
2007-01-11 14:30:32 +00:00
Jeff Squyres
add3909096 Back out 13076 and 13077 in favor of a much simpler approach.
Sorry for the configure change -- hopefully it's early enough in the
morning that it won't affect people... (new approach won't have a
configure change).

Refs trac:739.

This commit was SVN r13080.

The following Trac tickets were found above:
  Ticket 739 --> https://svn.open-mpi.org/trac/ompi/ticket/739
2007-01-11 14:07:15 +00:00
George Bosilca
ceb5d436d8 Missing an include.
This commit was SVN r13078.
2007-01-11 05:45:13 +00:00
George Bosilca
24a91fad1d OPAL_BOOL_STRUCT_COPY or OMPI_BOOL_STRUCT_COPY that's the question!
Let's minimize the disturbances and say that the configure system is right.
From now on it's OPAL_BOOL_STRUCT_COPY. This one is related to r13076 and
has to follow when r13076 goes in the 1.2.

This commit was SVN r13077.

The following SVN revision numbers were found above:
  r13076 --> open-mpi/ompi@f0932a0701
2007-01-11 05:44:48 +00:00
Jeff Squyres
f0932a0701 A workaround for a bug in the PGI 6.2 compiler series. This bug has
been fixed in the 7.0 PGI series, but is unlikely to be fixed in the
6.2 series:

 * Add a configure test looking for the bad behavior (the PGI compiler
   chokes on C code where structs containing bool's are copied by
   value)
 * Set OMPI_BOOL_STRUCT_COPY to 1 if it's ok, 0 if it's not (i.e., PGI
   6.2 series will have this value set to 0)
 * In two places in the code base -- orte-clean and btl_openib_ini.h,
   we have a struct that contains a bool that is copied by value.  In
   these two places, check OMPI_BOOL_STRUCT_COPY and if it's 1, use
   the "int" type instead of "bool".

Fixes trac:739

This commit was SVN r13076.

The following Trac tickets were found above:
  Ticket 739 --> https://svn.open-mpi.org/trac/ompi/ticket/739
2007-01-11 02:21:26 +00:00
Josh Hursey
93208445fd Make sure we wireup the 'verbose' MCA parameter for the BTL's.
This commit was SVN r13067.
2007-01-10 21:24:35 +00:00
Gleb Natapov
624f139bd8 This commit fixes trac:729. Initialize pointer to registration to NULL. Otherwise
it may contain garbage and we will try to unregister it later in btl_free().

This commit was SVN r13054.

The following Trac tickets were found above:
  Ticket 729 --> https://svn.open-mpi.org/trac/ompi/ticket/729
2007-01-09 10:29:20 +00:00
Gleb Natapov
d3ac56272a Prevent access to openib_btl after free().
This commit was SVN r13052.
2007-01-09 09:07:32 +00:00
Brian Barrett
e130f18cc2 Fix some compiler warnings that have slipped in lately...
This commit was SVN r13037.
2007-01-08 17:20:09 +00:00
Brian Barrett
a34e67d743 Remove unneeded PARAM_INIT_FILE variable in configure.params files used by
components that use configure.m4 for configuration or are always built. 
The macro has not been needed since moving to configure types other than
configure.stub

Fixes trac:590

This commit was SVN r13031.

The following Trac tickets were found above:
  Ticket 590 --> https://svn.open-mpi.org/trac/ompi/ticket/590
2007-01-08 03:44:22 +00:00