1
1
Граф коммитов

137 Коммитов

Автор SHA1 Сообщение Дата
Edgar Gabriel
100515e321 remove split collective interfaces from fcoll and their fake implemenations. Not required anymore 2015-06-18 05:20:07 -05:00
Edgar Gabriel
19cac73a9b first part of the changes trequired to support non-blocking colelctive io operations 2015-06-18 05:20:07 -05:00
Gilles Gouaillardet
0f17cdfc57 fcoll: fix misc memory leaks
as reported by Coverity with CIDs 72293,72294 and 1269894
2015-06-17 11:17:52 +09:00
Nathan Hjelm
033894b493 Merge pull request #541 from hjelmn/c99_components
C99 component initialization
2015-04-20 10:45:39 -06:00
Howard Pritchard
3339274136 Merge pull request #542 from hppritcha/topic/coverity_714118
fcoll/two_phase: coverity fix
2015-04-20 05:42:12 -06:00
Howard Pritchard
de215addc6 fcoll/two_phase: coverity fix
fix CID 714118

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-04-18 14:34:48 -06:00
Nathan Hjelm
df75d0382f ompi: use C99 subobject naming for component initialization
This commit helps future-proof ompi components by initializing each
component member by name.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-04-18 10:29:58 -06:00
Mangala Jyothi Bhaskar
c4de46e284 Fix number of aggregators used in two phase fcoll 2015-04-16 10:39:10 -05:00
Nathan Hjelm
3436f2917d Merge pull request #449 from hjelmn/mca_base_update
mca/base update
2015-04-16 08:41:48 -06:00
Howard Pritchard
05324e32ff fcoll/static: coverity fixes
Fix CIDs 72138, 72139, 72143

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-04-02 14:51:44 -06:00
Nathan Hjelm
b68d66bb9b MCA: Add the project/project version to the MCA base component
This commit adds support for project_framework_component_* parameter
matching. This is the first step in allowing the same framework name
in multiple projects. This change also bumps the MCA component version
to 2.1.0.

All master frameworks have been updated to use the new component
versioning macro. An mca.h has been added to each project to add a
project specific versioning macro of the form
PROJECT_MCA_VERSION_2_1_0.

Signed-off-by: Nathan Hjelm <hjelmn@me.com>
2015-03-27 10:59:04 -06:00
Howard Pritchard
66fee3bd18 fcoll/static: minor fix for coverity
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-11 09:11:49 -06:00
Howard Pritchard
fba88360a8 fcoll/dynamic: more coverity fixes
Okay coverity seems to get one stuck in a loop where
by fixing one set of resource allocation problems, it
starts finding more.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-09 15:01:05 -07:00
Howard Pritchard
2d61a652c8 fcoll/dynamic: coverity fixes
okay, hopefully really fix CIDS 72325-72328, and 72330-72332.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-09 13:53:52 -07:00
Howard Pritchard
209f002200 fcoll/static: fix an errant free
Got to excited about coverity and ended up generating
a new coverity error.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-06 13:12:53 -07:00
Howard Pritchard
4f4b99bbac fcoll/dynamic,static: coverity fixes
Fix some theoretical memory leaks reported by coverity.

Fixes CIDS 72332, 72328, 72332, 72138, 72139, 72140, 72364, 72365-72370
           72372-72374, 741354, 72143, 72375-83, 1027140, 1269903

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-06 11:05:23 -07:00
Nathan Hjelm
5f1254d710 Update code base to use the new opal_free_list_t
Use of the old ompi_free_list_t and ompi_free_list_item_t is
deprecated. These classes will be removed in a future commit.

This commit updates the entire code base to use opal_free_list_t and
opal_free_list_item_t.

Notes:

OMPI_FREE_LIST_*_MT -> opal_free_list_* (uses opal_using_threads ())

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-02-24 10:05:45 -07:00
Jeff Squyres
c7d8563d8d fcoll_two_phase: fix trivial compiler warning 2015-02-23 08:58:43 -08:00
Howard Pritchard
bf89131f9e add owner files to opa/ompi/orte mca directories
This commit adds an owner file in each of the component directories
for each framework.  This allows for a simple script to parse
the contents of the files and generate, among other things, tables
to be used on the project's wiki page.  Currently there are two
"fields" in the file, an owner and a status.  A tool to parse
the files and generate tables for the wiki page will be added
in a subsequent commit.
2015-02-22 15:10:23 -07:00
mjbhaskar
39f9880759 Fixed the data type argument in an all reduce operation to fix a bug
seen on 32 bit machines.
2015-01-08 14:18:54 -06:00
Edgar Gabriel
7e41e0e62b fix a segfault in the two-phase I/O algorithm for fileviews of 0 byte size. 2014-12-01 15:59:00 -06:00
Edgar Gabriel
0758d7570e part 1 of the fix to get rid of the missing symbols that prevent the sub-modules to be loaded. 2014-11-29 20:01:36 -06:00
Gilles Gouaillardet
64c18686b7 fix ompi_request_wait vs ompi_request_wait_all and
MPI_STATUS_IGNORE vs MPI_STATUSES_IGNORE
2014-11-04 12:16:30 +09:00
Edgar Gabriel
3a5f4f72da make the zero byte read/write scenarios work without the contiguous flag.
This commit was SVN r32690.
2014-09-09 16:26:14 +00:00
Edgar Gabriel
6a607caed8 fix some zero byte allocation scenarios.
This commit was SVN r32689.
2014-09-09 16:25:44 +00:00
Edgar Gabriel
ed02927767 - do not set the contiguous memory option in the collective operations. It
should not be stored on the file handle anyway, since it is not a property of
the file.
- protect a realloc for zero byte scenarios.

This commit was SVN r32678.
2014-09-07 18:09:43 +00:00
Edgar Gabriel
0f59ce6591 use the fbtl return value as originally intended, namely to retrieve the
number of bytes written and read. Status contains now the actual number of
bytes written for individual operations. For collective operations, this is
unfortunately not possible.

This commit was SVN r32674.
2014-09-07 15:14:57 +00:00
Edgar Gabriel
52eac0146d cleanup of the fbtl interfaces: remove the *sorted optimization flag, since it
was not used anyway in the last two years. Simplifies the code significantly.

This commit was SVN r32602.
2014-08-25 18:04:24 +00:00
Vishwanath Venkatesan
b176787d0f Remove unwanted spaces + Test commit
This commit was SVN r32576.
2014-08-22 05:11:17 +00:00
Ralph Castain
552c9ca5a0 George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT:    Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL

All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies.  This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP.  Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose.  UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs.  A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.

This commit was SVN r32317.
2014-07-26 00:47:28 +00:00
Edgar Gabriel
d4f83ab929 clean up of the MCA parameters of the fcoll framework. Most parameters are now
set/retrieved in ompio instead of the fcoll components.

This commit was SVN r32294.
2014-07-23 19:03:14 +00:00
Edgar Gabriel
217e61e345 add proper typcasts to intptr_t to avoid warnings on 32bit systems.
This commit was SVN r30229.
2014-01-10 16:19:04 +00:00
Brian Barrett
8b778903d8 Fix longstanding issue with our multi-project support. Rather than using
pkg{data,lib,includedir}, use our own ompi{data,lib,includedir}, which is
always set to {datadir,libdir,includedir}/openmpi.  This will keep us from
having help files in prefix/share/open-rte when building without Open MPI,
but in prefix/share/openmpi when building with Open MPI.

This commit was SVN r30140.
2014-01-07 22:11:15 +00:00
Edgar Gabriel
8ffc1aac89 update the _component.c files in ompio to use the explicit assignment of the
mca_register_component_params element of the structure.

This commit was SVN r28914.
2013-07-22 21:11:05 +00:00
Edgar Gabriel
93cef82873 remove the ylib component from the fcoll framework. It is not used, there are
no plans to use it. We can always recover it from svn if we would ever change
our minds.

This commit was SVN r28840.
2013-07-18 16:18:06 +00:00
Vishwanath Venkatesan
ce8f8f0829 Changing the MPI Datatype from MPI_LONG to OMPI_OFFSET_DATATYPE for send/recv offsets
This commit was SVN r28822.
2013-07-17 19:16:53 +00:00
Vishwanath Venkatesan
0b727f84da Avoid malloc of zero bytes, add a check and avoid it.
This commit was SVN r28597.
2013-06-06 14:08:57 +00:00
Vishwanath Venkatesan
7d6a05982a Removing the gather_array based on the flag UNIFORM FVIEW for read all operations (dynamic/static),
+ Disabling Timing data extraction by default in dynamic write all

This commit was SVN r28592.
2013-06-05 21:35:37 +00:00
Vishwanath Venkatesan
55878674d7 1. Removing the allgather_array based on the flag UNIFORM FVIEW. This is not really and optimization.
2. Fixing some of the debug printf's these are outdated.

This commit was SVN r28591.
2013-06-05 21:30:15 +00:00
Vishwanath Venkatesan
c902624b59 Using ompi_type_destroy to free ompi_datatype. This had to be updated in all the collective algorithms.
Hopefully this will fix all warnings.

This commit was SVN r28385.
2013-04-24 19:27:26 +00:00
Vishwanath Venkatesan
bba4a93f63 Got this wrong while replacing MPI function with OMPI functions. Fixed it now.
This commit was SVN r28350.
2013-04-22 19:58:25 +00:00
Vishwanath Venkatesan
53753622d4 Changing some of the MPI_ functions to ompi_ equivalents.
This commit was SVN r28342.
2013-04-17 21:06:36 +00:00
Vishwanath Venkatesan
74c418b860 Adding typecasting with intptr_t to remove warnings.
This commit was SVN r28278.
2013-04-03 17:07:43 +00:00
Vishwanath Venkatesan
784337aab1 typecasting with intptr_t to remove warnings
This commit was SVN r28276.
2013-04-03 17:06:02 +00:00
Nathan Hjelm
9d4a26f47d Update OMPI frameworks to use the MCA framework system.
Notes:
  - This commit also eliminates the need for an available components list in use
    in several frameworks. None of the code in question was making use of the
    priority field of the priority component list item so these extra lists were
    removed.
  - Cleaned up selection code in several frameworks to sort lists using opal_list_sort.
  - Cleans up the ompi/orte-info functions. Expose the functions that construct the
    list of params so they can be used elsewhere.

patches for mtl/portals4 from brian

missed a few output variables in openib

This commit was SVN r28241.
2013-03-27 21:17:31 +00:00
Nathan Hjelm
cf377db823 MCA/base: Add new MCA variable system
Features:
 - Support for an override parameter file (openmpi-mca-param-override.conf).
   Variable values in this file can not be overridden by any file or environment
   value.
 - Support for boolean, unsigned, and unsigned long long variables.
 - Support for true/false values.
 - Support for enumerations on integer variables.
 - Support for MPIT scope, verbosity, and binding.
 - Support for command line source.
 - Support for setting variable source via the environment using
   OMPI_MCA_SOURCE_<var name>=source (either command or file:filename)
 - Cleaner API.
 - Support for variable groups (equivalent to MPIT categories).

Notes:
 - Variables must be created with a backing store (char **, int *, or bool *)
   that must live at least as long as the variable.
 - Creating a variable with the MCA_BASE_VAR_FLAG_SETTABLE enables the use of
   mca_base_var_set_value() to change the value.
 - String values are duplicated when the variable is registered. It is up to
   the caller to free the original value if necessary. The new value will be
   freed by the mca_base_var system and must not be freed by the user.
 - Variables with constant scope may not be settable.
 - Variable groups (and all associated variables) are deregistered when the
   component is closed or the component repository item is freed. This
   prevents a segmentation fault from accessing a variable after its component
   is unloaded.
 - After some discussion we decided we should remove the automatic registration
   of component priority variables. Few component actually made use of this
   feature.
 - The enumerator interface was updated to be general enough to handle
   future uses of the interface.
 - The code to generate ompi_info output has been moved into the MCA variable
   system. See mca_base_var_dump().

opal: update core and components to mca_base_var system
orte: update core and components to mca_base_var system
ompi: update core and components to mca_base_var system

This commit also modifies the rmaps framework. The following variables were
moved from ppr and lama: rmaps_base_pernode, rmaps_base_n_pernode,
rmaps_base_n_persocket. Both lama and ppr create synonyms for these variables.

This commit was SVN r28236.
2013-03-27 21:09:41 +00:00
Vishwanath Venkatesan
e092cc34e0 Fixing the read all bugs discovered by Coverity
This commit was SVN r28189.
2013-03-20 20:27:09 +00:00
Ralph Castain
a4b6fb241f Remove all remaining vestiges of the Windows integration
This commit was SVN r28137.
2013-02-28 17:31:47 +00:00
Vishwanath Venkatesan
9320beaacc Modified/improved implmentation of dynamic segmentation algorithm to avoid merging in
fbtl modules. This implmentation in alignment with all other collective modules tries to 
keep all the file-ops as contiguous as possible.

This commit was SVN r27611.
2012-11-15 00:59:10 +00:00
Vishwanath Venkatesan
f91340f648 Fixes for the 2gb limitation. This fixes problems both static and two-phase
algorithms. 

This commit was SVN r27596.
2012-11-12 17:39:58 +00:00
Nathan Hjelm
bdedd8b0d3 Per RFC modify the behavior of mca_base_components_close to NOT close the output. Modify frameworks to always close their output and set to -1.
Reasoning: The old behavior was a little confusing. mca_base_components_open does not open an output stream so it is a little unexpected that mca_base_components_close does. To add to this several frameworks (that don't use mca_base_components_close) failed to close their output in the framework close function and others closed their output a second time. This change is an improvement to the symantics of mca_base_components_open/close as they are now symetric in their functionality.

This commit was SVN r27570.
2012-11-06 19:09:26 +00:00
Vishwanath Venkatesan
0e6378bfc9 Modifying the static component accordingly for the modification of interfaces in io_ompio.c
This commit was SVN r27546.
2012-10-31 22:09:21 +00:00
Vishwanath Venkatesan
d1fc22883a Changing the dynamic component accordingly for the modified interfaces
This commit was SVN r27545.
2012-10-31 22:08:25 +00:00
Vishwanath Venkatesan
67463de96f Changing the two-phase component accordingly for the modified interfaces.
This commit was SVN r27544.
2012-10-31 22:07:02 +00:00
Vishwanath Venkatesan
95d38fdaf5 # Extracting timing information for the static collective write/read algorithms.
# The processes register their information and continue.
# Actual printing of timing information happens at file close.
# Triggered by MCA parameter at runtime

This commit was SVN r27442.
2012-10-11 21:27:47 +00:00
Vishwanath Venkatesan
240d56feeb # Extracting timing information for the two-phase collective write/read algorithms.
# The processes register their information and continue.
# Actual printing of timing information happens at file close.
# Triggered by MCA parameter at runtime

This commit was SVN r27441.
2012-10-11 21:25:30 +00:00
Vishwanath Venkatesan
7bc35f7862 # Extracting timing information for the dynamic collective write/read algorithms.
# The processes register their information and continue.
# Actual printing of timing information happens at file close.
# Triggered by MCA parameter at runtime

This commit was SVN r27440.
2012-10-11 21:23:24 +00:00
Vishwanath Venkatesan
c86e5f6263 set status->_ucount correctly for two-phase collective read and write operations in the module
This commit was SVN r27406.
2012-10-04 21:39:37 +00:00
Vishwanath Venkatesan
76dc8a7c55 Fix for two_phase_read_all for multiple cycles.
This commit was SVN r27388.
2012-10-01 19:14:14 +00:00
Vishwanath Venkatesan
c6751eaf70 Fixing all two_phase read_all bugs,
1. Multiple aggregator with non-contiguous datatype,
2. Memory corruption bugs.

Cleaned version, with proper initialization and memory management.

This commit was SVN r27370.
2012-09-26 00:16:08 +00:00
Vishwanath Venkatesan
2e2a46b2be Fixing two-phase write all bug for non-contiguous file-type.
This commit was SVN r27369.
2012-09-26 00:10:40 +00:00
Vishwanath Venkatesan
6d9d0f2968 Initialize the iov_count, this crashes static write/read in certain platforms while decoding datatype
This commit was SVN r27273.
2012-09-08 00:40:21 +00:00
Vishwanath Venkatesan
bf58af295b Changes to the two_phase implementation, for supporting the
data-seiving feature of two-phase algorithm. 

This commit was SVN r27152.
2012-08-27 21:11:05 +00:00
Vishwanath Venkatesan
539571171b Moving support functions of two-phase to the two_phase component.
This commit was SVN r26920.
2012-07-31 04:37:04 +00:00
Edgar Gabriel
1078f13ad2 set status->_ucount correctly for collective read and write operations in the module
This commit was SVN r26916.
2012-07-30 20:14:36 +00:00
Edgar Gabriel
81a050add9 simplify the individual fcoll module by just calling the
mca_io_ompio_file_read/write functions directly. Avoid replicating the code in 
both places.

This commit was SVN r26909.
2012-07-30 15:44:22 +00:00
Vishwanath Venkatesan
86a57c7b66 Initializing sorted_file_offsets to NULL
This commit was SVN r26526.
2012-05-30 06:56:40 +00:00
Vishwanath Venkatesan
cbad31cc88 1. Freeing the displs array after allgatherv to avoid segmentation faults in dynamic segmentation
2. Checking for 0 bytes datatypes  and sending only when data available to avoid 0 byte messages being sent and received. 
3. Changing timing extraction to support calculating, min, max and avg communication costs + min and avg write costs

This commit was SVN r26450.
2012-05-18 21:39:58 +00:00
Edgar Gabriel
636cf786be local_heap_sort should really be a static function.
This commit was SVN r26065.
2012-02-28 14:42:56 +00:00
Vishwanath Venkatesan
7c9c3ede61 Modified implementation for the static segmentation read algorithm with improved
performance and bug fixes.

This commit was SVN r26056.
2012-02-24 20:55:33 +00:00
Vishwanath Venkatesan
d5a9223a9a Removed a variable which was allocated but never used.
This commit was SVN r26055.
2012-02-24 20:48:52 +00:00
Vishwanath Venkatesan
326bc69df4 Modified implementation for static file write all algorithm which fixes all the previous
bugs and provides improved performance.

This commit was SVN r26054.
2012-02-24 20:47:09 +00:00
Vishwanath Venkatesan
15ebe838e9 Modified implementation of two_phase read all similar to the changes for the write all
incorporating romio style partitioning.

This commit was SVN r25853.
2012-02-01 18:30:13 +00:00
Vishwanath Venkatesan
158374bdd0 Dynamic file write all algorithm optimized by using derived datatype for receiving actual data
thereby avoiding the merging step in the fbtl.

This commit was SVN r25852.
2012-02-01 18:20:44 +00:00
Vishwanath Venkatesan
b9026ccbd0 Fix for two-phase generating flattened datatype using decoded iovec for
handling non-contiguous memory and contiguous file cases.

This commit was SVN r25850.
2012-02-01 17:23:51 +00:00
Jeff Squyres
4f35b62154 Components should not be linking to top-level libraries -- and
definitely should not be linking to more than libmpi.la!  (remember
that libmpi.la now wholly contains libopen-rte.la, which wholly
contains libopen-pal.la).

This commit was SVN r25843.
2012-01-31 20:43:27 +00:00
Vishwanath Venkatesan
1e95d8b1e2 remove the MPI functions used in these files by the OMPI internal corresponding functionality and also add error checking in these for functions which did not have them'
This commit was SVN r25723.
2012-01-13 17:21:51 +00:00
Edgar Gabriel
e0139a2d7e provide descriptions about the functionality of these frameworks.
This commit was SVN r25682.
2011-12-22 19:42:00 +00:00
Vishwanath Venkatesan
0f928be1d5 Modifying selection logic back to select two-phase at the cases it should.
This commit was SVN r25681.
2011-12-22 01:01:32 +00:00
Vishwanath Venkatesan
37c8470e3d modified implementation for two-phase write_all incorporating romio style domain partitioning
This commit was SVN r25680.
2011-12-22 00:16:29 +00:00
Jeff Squyres
c10f41c87e Do not build these frameworks when --disable-mpi-io is specified.
Fixes some Cisco MTT MPI install errors.

This commit was SVN r25566.
2011-12-02 22:11:23 +00:00
George Bosilca
ce7935c8fa Obviously these were not needed.
This commit was SVN r25231.
2011-10-04 14:56:34 +00:00
Vishwanath Venkatesan
2ee2b478d8 Modifying selection logic to select dynamic for cases where two_phase
fails.

This commit was SVN r25171.
2011-09-20 21:57:23 +00:00
Shiqing Fan
16193771ba Add one missing header file. Fix the MTT build for Windows.
This commit was SVN r25112.
2011-08-31 13:15:05 +00:00
Edgar Gabriel
f46ef05c6e ompi_ignore some components that depend on the configure logic, since some
libs don't seem to propagate correctly under certain circumstances. This makes
hopefully the nightly tests pass.

also, remove the files that should not have been committed in the first place
:-)

This commit was SVN r25085.
2011-08-26 00:49:32 +00:00
Edgar Gabriel
61ac1dbcf3 silence some warnings.
This commit was SVN r25080.
2011-08-25 21:22:34 +00:00
Edgar Gabriel
52063267df commit of the OMPIO modules and frameworks.
This commit was SVN r25079.
2011-08-25 20:08:17 +00:00