1
1
Граф коммитов

143 Коммитов

Автор SHA1 Сообщение Дата
Shiqing Fan
a5281f0434 - 1/4 commit for Windows Visual Studio and CCP support:
CMakeLists and .windows files.
  In contribs preconfigured and precompiled parts.

This commit was SVN r20108.
2008-12-10 20:59:20 +00:00
George Bosilca
82d1d5d785 The patch for "Unexpected message queue for unknown CID's required" ticket #1460.
I'm unable to split it in two parts, my patch and Edgar's one. So I just update
copyright information for both of us.
What this patch do:
- it use the unexpected queue create by commit r19562 to dispatch the
  unexpected message to the right communicator (once this communicator
  is created and initialized).
- delay the PML comm_add until we have the context_id for the new communicator.
- only do the PML comm_add on processes that really belong to the new
  communicator. Please read the lengthy comment in the source code for the
  reason behind this.

This commit was SVN r19929.

The following SVN revision numbers were found above:
  r19562 --> open-mpi/ompi@acd3406aa7
2008-11-04 21:58:06 +00:00
George Bosilca
00d24bf8ab Scalability patch, or slim-fast effect #1. All BML structures just
got a whole lot smaller, decreasing the memory footprint of the
running application. How much it's a good question. Here is a
breakdown:

- in mca_bml_base_endpoint_t: 3 *size_t + 1 * uint32_t
- in mca_bml_base_btl_t: 1 * int + 1 * double - 1 * float
                         + 6 * size_t + 9 * (void*)

The decrease in mca_bml_base_endpoint_t is for each peer and the
decrease in mca_bml_base_btl_t is for each BTL for each peer.
So, if we consider the most convenient case where there is only
one network between all peers, this decrease the memory foot print
per peer by
9*size_t + 9*(void*) + 2 * int32_t + 1 * double - 1 * float.
On a 64 bits machine this will be 156 bytes per peer.

Now we access all these fields directly from the underlying BTL
structure, and as this structure is common to multiple BML endpoint,
we are a lot more cache friendly. Even if this do not improve the
latency, it makes the SM performance graph a lot smoother.

This commit was SVN r19659.
2008-09-30 21:02:37 +00:00
Shiqing Fan
08f54758bc - The MS VC compiler complains "error C2512: 'ompi_convertor_master_t' : no appropriate default constructor available", initialize it, it's not the fast path anyway.
This commit was SVN r19433.
2008-08-27 15:57:59 +00:00
Jeff Squyres
0af7ac53f2 Fixes trac:1392, #1400
* add "register" function to mca_base_component_t
   * converted coll:basic and paffinity:linux and paffinity:solaris to
     use this function
   * we'll convert the rest over time (I'll file a ticket once all
     this is committed)
 * add 32 bytes of "reserved" space to the end of mca_base_component_t
   and mca_base_component_data_2_0_0_t to make future upgrades
   [slightly] easier
   * new mca_base_component_t size: 196 bytes
   * new mca_base_component_data_2_0_0_t size: 36 bytes
 * MCA base version bumped to v2.0
   * '''We now refuse to load components that are not MCA v2.0.x'''
 * all MCA frameworks versions bumped to v2.0
 * be a little more explicit about version numbers in the MCA base
   * add big comment in mca.h about versioning philosophy

This commit was SVN r19073.

The following Trac tickets were found above:
  Ticket 1392 --> https://svn.open-mpi.org/trac/ompi/ticket/1392
2008-07-28 22:40:57 +00:00
Rolf vandeVaart
9c080b27d6 Fix for bug when running 64-bit heterogeneous.
This commit fixes trac:1341.

This commit was SVN r18940.

The following Trac tickets were found above:
  Ticket 1341 --> https://svn.open-mpi.org/trac/ompi/ticket/1341
2008-07-17 19:04:40 +00:00
Terry Dontje
12baa72580 This commit fixes trac:1306
This commit was SVN r18718.

The following Trac tickets were found above:
  Ticket 1306 --> https://svn.open-mpi.org/trac/ompi/ticket/1306
2008-06-24 14:38:11 +00:00
Ralph Castain
9613b3176c Effectively revert the orte_output system and return to direct use of opal_output at all levels. Retain the orte_show_help subsystem to allow aggregation of show_help messages at the HNP.
After much work by Jeff and myself, and quite a lot of discussion, it has become clear that we simply cannot resolve the infinite loops caused by RML-involved subsystems calling orte_output. The original rationale for the change to orte_output has also been reduced by shifting the output of XML-formatted vs human readable messages to an alternative approach.

I have globally replaced the orte_output/ORTE_OUTPUT calls in the code base, as well as the corresponding .h file name. I have test compiled and run this on the various environments within my reach, so hopefully this will prove minimally disruptive.

This commit was SVN r18619.
2008-06-09 14:53:58 +00:00
Ralph Castain
c992e99035 Remove the tags from orte_output_open and the filtering operation from orte_output - this will be handled differently to improve the XML output interface
This commit was SVN r18557.
2008-06-03 14:24:01 +00:00
George Bosilca
e361bcb64c Send optimizations.
1. The send path get shorter. The BTL is allowed to return > 0 to specify that the
   descriptor was pushed to the networks, and that the memory attached to it is 
   available again for the upper layer. The MCA_BTL_DES_SEND_ALWAYS_CALLBACK flag
   can be used by the PML to force the BTL to always trigger the callback.
   Unmodified BTL will continue to work as expected, as they will return OMPI_SUCCESS
   which force the PML to have exactly the same behavior as before. Some BTLs have
   been modified: self, sm, tcp, mx.
2. Add send immediate interface to BTL.
   The idea is to have a mechanism of allowing the BTL to take advantage of
   send optimizations such as the ability to deliver data "inline". Some
   network APIs such as Portals allow data to be sent using a "thin" event
   without packing data into a memory descriptor. This interface change
   allows the BTL to use such capabilities and allows for other optimizations
   in the future. All existing BTLs except for Portals and sm have this interface
   set to NULL.

This commit was SVN r18551.
2008-05-30 03:58:39 +00:00
Jeff Squyres
e7ecd56bd2 This commit represents a bunch of work on a Mercurial side branch. As
such, the commit message back to the master SVN repository is fairly
long.

= ORTE Job-Level Output Messages =

Add two new interfaces that should be used for all new code throughout
the ORTE and OMPI layers (we already make the search-and-replace on
the existing ORTE / OMPI layers):

 * orte_output(): (and corresponding friends ORTE_OUTPUT,
   orte_output_verbose, etc.)  This function sends the output directly
   to the HNP for processing as part of a job-specific output
   channel.  It supports all the same outputs as opal_output()
   (syslog, file, stdout, stderr), but for stdout/stderr, the output
   is sent to the HNP for processing and output.  More on this below.
 * orte_show_help(): This function is a drop-in-replacement for
   opal_show_help(), with two differences in functionality:
   1. the rendered text help message output is sent to the HNP for
      display (rather than outputting directly into the process' stderr
      stream)
   1. the HNP detects duplicate help messages and does not display them
      (so that you don't see the same error message N times, once from
      each of your N MPI processes); instead, it counts "new" instances
      of the help message and displays a message every ~5 seconds when
      there are new ones ("I got X new copies of the help message...")

opal_show_help and opal_output still exist, but they only output in
the current process.  The intent for the new orte_* functions is that
they can apply job-level intelligence to the output.  As such, we
recommend that all new ORTE and OMPI code use the new orte_*
functions, not thei opal_* functions.

=== New code ===

For ORTE and OMPI programmers, here's what you need to do differently
in new code:

 * Do not include opal/util/show_help.h or opal/util/output.h.
   Instead, include orte/util/output.h (this one header file has
   declarations for both the orte_output() series of functions and
   orte_show_help()).
 * Effectively s/opal_output/orte_output/gi throughout your code.
   Note that orte_output_open() takes a slightly different argument
   list (as a way to pass data to the filtering stream -- see below),
   so you if explicitly call opal_output_open(), you'll need to
   slightly adapt to the new signature of orte_output_open().
 * Literally s/opal_show_help/orte_show_help/.  The function signature
   is identical.

=== Notes ===

 * orte_output'ing to stream 0 will do similar to what
   opal_output'ing did, so leaving a hard-coded "0" as the first
   argument is safe.
 * For systems that do not use ORTE's RML or the HNP, the effect of
   orte_output_* and orte_show_help will be identical to their opal
   counterparts (the additional information passed to
   orte_output_open() will be lost!).  Indeed, the orte_* functions
   simply become trivial wrappers to their opal_* counterparts.  Note
   that we have not tested this; the code is simple but it is quite
   possible that we mucked something up.

= Filter Framework =

Messages sent view the new orte_* functions described above and
messages output via the IOF on the HNP will now optionally be passed
through a new "filter" framework before being output to
stdout/stderr.  The "filter" OPAL MCA framework is intended to allow
preprocessing to messages before they are sent to their final
destinations.  The first component that was written in the filter
framework was to create an XML stream, segregating all the messages
into different XML tags, etc.  This will allow 3rd party tools to read
the stdout/stderr from the HNP and be able to know exactly what each
text message is (e.g., a help message, another OMPI infrastructure
message, stdout from the user process, stderr from the user process,
etc.).

Filtering is not active by default.  Filter components must be
specifically requested, such as:

{{{
$ mpirun --mca filter xml ...
}}}

There can only be one filter component active.

= New MCA Parameters =

The new functionality described above introduces two new MCA
parameters:

 * '''orte_base_help_aggregate''': Defaults to 1 (true), meaning that
   help messages will be aggregated, as described above.  If set to 0,
   all help messages will be displayed, even if they are duplicates
   (i.e., the original behavior).
 * '''orte_base_show_output_recursions''': An MCA parameter to help
   debug one of the known issues, described below.  It is likely that
   this MCA parameter will disappear before v1.3 final.

= Known Issues =

 * The XML filter component is not complete.  The current output from
   this component is preliminary and not real XML.  A bit more work
   needs to be done to configure.m4 search for an appropriate XML
   library/link it in/use it at run time.
 * There are possible recursion loops in the orte_output() and
   orte_show_help() functions -- e.g., if RML send calls orte_output()
   or orte_show_help().  We have some ideas how to fix these, but
   figured that it was ok to commit before feature freeze with known
   issues.  The code currently contains sub-optimal workarounds so
   that this will not be a problem, but it would be good to actually
   solve the problem rather than have hackish workarounds before v1.3 final.

This commit was SVN r18434.
2008-05-13 20:00:55 +00:00
Shiqing Fan
f35a06119c Use memchecker_convertor_call function instead the old one. Move the function to the place that we can use convertor.
This commit was SVN r18370.
2008-05-05 13:57:27 +00:00
Ralph Castain
fa082cafa9 Shift the architecture calculation from the ompi/datatype engine to the opal/util area. This allows us to compute the architecture earlier in the launch and communicate it outside of the modex.
Note: this is an early preliminary step in the movement of portions of the datatype engine to the opal layer.

This commit was SVN r18198.
2008-04-17 20:43:56 +00:00
Shiqing Fan
1c4c7e0f2f Add memchecker support for osc rdma communication.
This commit was SVN r18173.
2008-04-16 13:29:55 +00:00
Shiqing Fan
79da2fdd2c Use the new memchecker convertor function.
Remove some unnecessary memchecker calls.

This commit was SVN r18172.
2008-04-16 13:24:35 +00:00
Shiqing Fan
54c7b71cfd Use the correct way of including memchecker.h, which will work with '--with-devel-headers'.
This commit was SVN r17435.
2008-02-12 18:01:17 +00:00
Shiqing Fan
f5792bbda5 merging the memchecker into trunk.
This commit was SVN r17424.
2008-02-12 08:46:27 +00:00
Tim Prins
b88a3f7a94 Update onesided components to fix the case (on 64 bit machines) where the total offset is greater than 2^31-1 bytes.
See: http://www.open-mpi.org/community/lists/users/2008/01/4880.php

This commit was SVN r17400.
2008-02-07 18:45:35 +00:00
Jeff Squyres
213b5d5c6e Per long threads on the mailing list and much confusion discussion
about linkers, have all OPAL, ORTE, and OMPI components '''not'' link
against the OPAL, ORTE, or OMPI libraries.

See ttp://www.open-mpi.org/community/lists/users/2007/10/4220.php for
details (or https://svn.open-mpi.org/trac/ompi/wiki/Linkers for a
better-formatted version of the same info).

This commit was SVN r16968.
2007-12-15 13:32:02 +00:00
Gleb Natapov
e2e211f23b Add flags parameter to btl_alloc() and btl_prepare_src() functions. If BTL
knows at the time of allocation priority of a descriptor it may do some
optimizations.

This commit was SVN r16901.
2007-12-09 14:08:01 +00:00
Gleb Natapov
04578ffdd6 Change calls to bml_btl->btl_alloc() to mca_bml_base_alloc().
This commit was SVN r16596.
2007-10-28 16:04:17 +00:00
Shiqing Fan
a0660f4deb - Just some type casts.
This commit was SVN r16100.
2007-09-12 15:29:58 +00:00
Shiqing Fan
efdcfa3807 - "extern 'C'" has been set twice. Remove one.
This commit was SVN r16022.
2007-08-30 15:03:59 +00:00
Jeff Squyres
466394a878 We only care about the value of ret in the
!OMPI_ENABLE_PROGRESS_THREADS case.  Reviewed by Brian.

This commit was SVN r16000.
2007-08-29 01:36:17 +00:00
Brian Barrett
af4e86c25f Update collectives selection logic to allow for multiple components to be
used at nce (up to one unique collective module per collective function).
Matches r15795:15921 of the tmp/bwb-coll-select branch

This commit was SVN r15924.

The following SVN revisions from the original message are invalid or
inconsistent and therefore were not cross-referenced:
  r15795
  r15921
2007-08-19 03:37:49 +00:00
George Bosilca
c961cb5749 The Windows support is now back in bussiness.
This commit was SVN r15599.
2007-07-25 03:55:34 +00:00
Brian Barrett
9b14008f61 add a couple of comments, clean up the organization a bit
This commit was SVN r15499.
2007-07-18 22:56:33 +00:00
Brian Barrett
6a1f876e98 Don't inline this function so that we can access the predefined datatype
array even when visibility is turned on

This commit was SVN r15444.
2007-07-16 16:29:51 +00:00
Brian Barrett
c9ad5d1f24 ooops, need to handle case where extents are not same as type sizes
This commit was SVN r15423.
2007-07-13 21:26:12 +00:00
Brian Barrett
7a9a8c7e17 Support reduction operations other than MPI_REPLACE for user-defined
datatypes with MPI_ACCUMULATE

This commit was SVN r15418.
2007-07-13 20:46:12 +00:00
Brian Barrett
1f2942cf2a * Provide flag if the BTL can do RDMA, but requires a prepare_{src,dst}
that exactly describes the buffer to be used as the target of the
    operation
  * Use the above flag to disable components setting the flag from being
    used for real RDMA operations for the one-sided component (the
    BTLs will still be used for RDMA transfers for the PML and for
    send/receive communication for the OSC component) 

This commit was SVN r15375.
2007-07-11 21:21:40 +00:00
Brian Barrett
739fed9dc9 Don't poke at internal structure fiealds of communicators or groups, but
instead use accessor functions

This commit was SVN r15366.
2007-07-11 17:16:06 +00:00
Brian Barrett
25e52238ab add ability to buffer put/accumulate messages during an epoch
This commit was SVN r15295.
2007-07-05 21:40:06 +00:00
Brian Barrett
5bbee1482e make debugging output slightly more useful
This commit was SVN r15294.
2007-07-05 21:01:32 +00:00
Brian Barrett
6c9de88d13 while the BTL semantics are sorted out, wait for completion of all rdma events before starting the ack
This commit was SVN r15292.
2007-07-05 16:50:05 +00:00
Brian Barrett
74008aac53 Support real RDMA operations for networks that support it
This commit was SVN r15288.
2007-07-05 03:32:32 +00:00
Brian Barrett
f5c721d11c Wire up all the RDMA-capable BTLs. Still no RDMA communication, but the
datastructures are finally all there

This commit was SVN r15271.
2007-07-02 22:22:59 +00:00
Brian Barrett
e279192865 ug - fix some dumb copy-n-paste errors
This commit was SVN r15188.
2007-06-25 01:59:34 +00:00
Brian Barrett
42b2c4e1df * RELEASE not DETRUCT things created with OBJ_NEW to fix a memory leak
* Fix potential race condition with starting a new lock epoch if we
   were releasing a lock
 * Increment the shared counter if we start a shared lock session during
   the unlock code

This commit was SVN r15186.
2007-06-24 23:30:10 +00:00
Brian Barrett
2ed0548da8 * No need for waiting until exposure epochs are over in order to complete
a WIN_FREE
  * Fix race condition in threaded builds with pending unlocks and
    finishing an epoch
  * Fix memory leak due to use of OBJ_DESTRUCT instead of OBJ_RELEASE
  * Fix race condition between releasing multiple shared locks and
    starting a new lock
  * Need to incremement the shared count if starting a new shared
    lock once an exclusive lock finishes

This commit was SVN r15185.
2007-06-24 22:36:00 +00:00
Brian Barrett
5528e0ca60 Properly initialize variable for threaded case
This commit was SVN r15174.
2007-06-22 15:29:06 +00:00
Brian Barrett
5f16251808 revert r15167. I don't know what I was thinking, but it was most definitely
"not right".

This commit was SVN r15172.

The following SVN revision numbers were found above:
  r15167 --> open-mpi/ompi@faa401dc47
2007-06-22 15:25:39 +00:00
Brian Barrett
8031f6561e Make a bunch of debugging calls use the macro version
This commit was SVN r15170.
2007-06-21 22:24:40 +00:00
Brian Barrett
80c50120ad debugging output should be macro version
This commit was SVN r15168.
2007-06-21 22:09:37 +00:00
Brian Barrett
faa401dc47 * Need to OBJ_RELEASE, not OBJ_DESTRUCT things that were created with
OBJ_NEW
  * Need to single when the passive unlock has left an expose epoch for
    the win_free case
  * Clean up some debugging output
  * fix missing variable initialization

This commit was SVN r15167.
2007-06-21 22:08:30 +00:00
Brian Barrett
0798c0784d properly set fields so that most difficult alignment rules are always met.
This commit was SVN r14854.
2007-06-05 01:46:04 +00:00
Brian Barrett
8cf02de3b4 * cleanup some ompi_info output
* enable eager sending by default

This commit was SVN r14813.
2007-05-30 22:23:34 +00:00
Brian Barrett
a2713dcac8 eeks! Bad to notice after committing the pt2pt part of r14806 that the
compile failed because of the wrong variable name.

This commit was SVN r14807.

The following SVN revision numbers were found above:
  r14806 --> open-mpi/ompi@7e57bbb0ef
2007-05-30 20:33:08 +00:00
Brian Barrett
7e57bbb0ef React slightly better when datatype creation from a buffer fails
This commit was SVN r14806.
2007-05-30 20:32:02 +00:00
Brian Barrett
84f7ed70b3 Re-enable the ability for the rdma one-sided component to start messages
as soon as the epochs allow, rather than waiting for the end of the
synchronization phase.

This commit was SVN r14800.
2007-05-30 17:06:19 +00:00