1
1
Граф коммитов

378 Коммитов

Автор SHA1 Сообщение Дата
Edgar Gabriel
f97655f28e make sure the iov buffer is initialized to zero, otherwise bad things can happen for 0-byte contributions on a process. 2015-10-15 12:46:01 -05:00
William Throwe
80bb41a079 ROMIO configure looks for lstat in wrong header
ROMIO configure looks for lstat in wrong header

The ROMIO configure script checks for a declaration of lstat in
unistd.h, but, at least on the Linux machines I checked, lstat is in
sys/stat.h.  (The detection failure led to a linker error when building
ROMIO as part of OpenMPI on one of my admittedly strangely configured
machines, somehow.)  It appears from the man page that either location
is possible, so check both.

(cherry picked from mpich/mpich@7b8bd055df)

Signed-off-by: Rob Latham <robl@mcs.anl.gov>
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-09-23 11:56:53 -06:00
Edgar Gabriel
3734a38370 this file should have been part of the previous commit. for removeing io_ompio_nbc.[ch] 2015-09-18 09:28:25 -05:00
Edgar Gabriel
cf46a6bd4d remove the io_ompio_nbc.[ch] files, they are not used anymore at this point in time. 2015-09-18 09:26:25 -05:00
Gilles Gouaillardet
fe351f6801 io: do not cast way the const modifier when this is not necessary
update the io framework and mpi c bindings
2015-09-09 09:18:58 +09:00
Nathan Hjelm
f451876058 Merge pull request #825 from hjelmn/white_space_purge
periodic trailing whitespace purge
2015-08-25 19:23:52 -06:00
Edgar Gabriel
70078175ee fix coverty warning 72107 2015-08-25 09:23:37 -05:00
Edgar Gabriel
a73f9470e0 fix coverty warning 1269829 2015-08-25 09:22:48 -05:00
Nathan Hjelm
156ce6af21 periodic whitespace purge
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-08-24 09:32:33 -06:00
Edgar Gabriel
b0461f8d3c the back pointer from the ompio_file structure to the ompi_file_t structure
has to be set earlier in case the user disables the lazy_open option.
2015-08-19 17:11:42 -05:00
Edgar Gabriel
7e370948c1 first cut on the fileview for shared filepointers fix. 2015-08-19 17:11:42 -05:00
Edgar Gabriel
072b18e197 Code cleanup for the time breakdown feature in ompio/fcoll
- make the internal structure follow the Open MPI naming convention
 - provide a single flag/macro which controls the compilation/utilization of this
   feature, to avoid that somebody using this has to modify every single
   fcoll component. A configure option could be added later if desired.
2015-08-14 08:53:04 -05:00
Edgar Gabriel
4bfc6ae798 Performance tuning: incorporate the usage of non-blocking operations in our array group-communication operations. 2015-08-13 20:05:18 -05:00
Edgar Gabriel
9e29edf15c remove a erroneous paranthesis which prevents the compilation of the lustre adio 2015-08-07 15:22:41 -05:00
Edgar Gabriel
0aa3049bfc Performance tuning: change the default behavior of ompio to *not* segment individual read/write operations.
In most cases, performance seems to be better if not segmented.
2015-08-07 13:06:39 -05:00
Edgar Gabriel
db5af26de7 Performance tuning. make sure we catch if the user wants to set the default fileview and replace it with our optimized default file view. Otherwise, performance will suffer. file_get_view should still return the correct filetype, not our optimized default file view. This is the correct version compared to ffa67b9693, which unfortunately broke
some test cases in mpi_test_suite. Thanks for @ggouaillardet for reporting this!
2015-08-07 12:49:58 -05:00
Edgar Gabriel
6f6c01ee8d free the datatypes that were created using type_dup during file_set_view 2015-08-07 11:50:25 -05:00
Edgar Gabriel
1ae4f8c7e6 Revert "Performance tuning. make sure we catch if the user wants to set the default fileview and replace it with"
This reverts commit ffa67b9693.
2015-08-07 09:53:07 -05:00
Howard Pritchard
8649a9f6ef Merge pull request #757 from roblatham00/lustre-excl-open-fix
hint processing should not open files
2015-07-31 12:16:14 -06:00
Edgar Gabriel
ffa67b9693 Performance tuning. make sure we catch if the user wants to set the default fileview and replace it with
our optimized default file view. Otherwise, performance will suffer. file_get_view should still return the correct filetype, not our optimized default file view
2015-07-30 19:15:00 -05:00
Edgar Gabriel
9b2a7e41f0 make sure the final number of aggregators is recorded correctly when not using
our aggregator selection logic.
2015-07-30 17:24:01 -05:00
Rob Latham
6e9cbe397f hint processing should not open files
move opening of files from hint processing and into open routines.

This is MPICH commit 92f1c69f0de8 and 22a77dceda11

see https://trac.mpich.org/projects/mpich/ticket/2261
Ref: https://github.com/open-mpi/ompi/issues/158

Signed-off-by: Pavan Balaji <balaji@anl.gov>
2015-07-30 12:25:20 -05:00
Edgar Gabriel
824d488709 - make the split collective shared file pointer operations work
- minor code restructering in io/ompio required for that.
2015-07-28 09:05:05 -05:00
Edgar Gabriel
e380f8c235 - fix the delete priority of the ompio component
- some application use MPI_File_delete as a collective function (e.g. IOR), which I think is not really covered by the standard. Right now, one process succeeds and theother ones return an error code. Fix that by not returning no error if the file that we try to delete does not exist anymore, to make these applications work.
2015-07-27 15:53:40 -05:00
Edgar Gabriel
3fb0614566 mark the request as ACTIVE 2015-07-27 12:43:45 -05:00
Edgar Gabriel
b484784dca make ompio return gracefully in case something goes wrong early in file_open. 2015-07-20 10:03:16 -05:00
Edgar Gabriel
86c3000e18 fix the delete selection logic in io/base. With the previous version, there was a mismatch
in the version number and no component was selected for file_delete.
2015-07-20 10:01:30 -05:00
Edgar Gabriel
f2af8e94ff - first cut on the io interface changes
- add the C interfaces for the new non-blocking collective I/O functions of MPI 3.1
2015-07-09 10:58:13 -05:00
Gilles Gouaillardet
9e89985f3d restore whitespaces into the pdf files 2015-07-07 09:17:00 +09:00
Nathan Hjelm
4d92c9989e more c99 updates
This commit does two things. It removes checks for C99 required
headers (stdlib.h, string.h, signal.h, etc). Additionally it removes
definitions for required C99 types (intptr_t, int64_t, int32_t, etc).

Signed-off-by: Nathan Hjelm <hjelmn@me.com>
2015-06-25 10:14:13 -06:00
Ralph Castain
869041f770 Purge whitespace from the repo 2015-06-23 20:59:57 -07:00
Edgar Gabriel
dedeee9771 finishing the changes for the non-blocking and split cpllective I/O operations. Everything except for the
interface changes to the io framework is done.
2015-06-18 06:22:41 -05:00
Edgar Gabriel
3b11a8b61c making the current work compile. 2015-06-18 05:56:51 -05:00
Edgar Gabriel
cc219281ba checkpoint of the current work, since I need to resync wioth master to fix the compilation problems 2015-06-18 05:20:07 -05:00
Edgar Gabriel
100515e321 remove split collective interfaces from fcoll and their fake implemenations. Not required anymore 2015-06-18 05:20:07 -05:00
Gilles Gouaillardet
0f08070a1c ompio: fix misc memory leaks
as identified by Coverity with CIDs 72147-72149, 731275 and 1269872
2015-06-17 11:17:54 +09:00
Edgar Gabriel
aa72e5b2ca fix the selection logic to not overwrite on the new aggregator side the list of
aggregators determined by the algorithm.
2015-05-27 22:35:45 -05:00
Gilles Gouaillardet
b6c67e051d io/ompio: fix misc memory leaks
as reported by Coverity with CIDs 72147-72149,72187,72188,731274,731275,741356,
1269889,1269893,1271535 and 1269872
2015-05-20 17:19:39 +09:00
Gilles Gouaillardet
650289bc33 romio314: update one more romio->romio314 name
Also missed this in open-mpi/ompi@db257cdbc0.
2015-05-08 18:26:33 +09:00
Gilles Gouaillardet
9d56b85b55 initialize common symbols from ompi 2015-05-08 10:11:58 +09:00
Gilles Gouaillardet
ab148e4e0c romio314: update one more romio->romio314 name
Also missed this in open-mpi/ompi@db257cdbc0.
2015-05-08 09:12:22 +09:00
Jeff Squyres
b3d89cf7b0 romio314: update one more romio->romio314 name
Missed this in db257cdbc0.
2015-05-07 09:40:45 -07:00
Jeff Squyres
691b4ec1e5 romio314: whitespace cleanup
No code changes
2015-05-05 06:23:59 -07:00
Jeff Squyres
db257cdbc0 romio314: adhere to the prefix rule
Rename all files and symbols from "io_romio" to "io_romio314".  This
fixes --disable-dlopen builds (because they were missing
the mca_io_romio314_component symbol).
2015-05-05 06:23:59 -07:00
Jeff Squyres
8127c24f30 romio314/Makefile.am: whitespace cleanup
No code changes.
2015-05-04 07:20:11 -07:00
Jeff Squyres
332bca7183 romio314/Makefile.am: name the component library properly 2015-05-04 07:20:11 -07:00
Gilles Gouaillardet
6b3126e69e ROMIO 3.1.4 refresh: add refresh notes 2015-04-30 19:02:20 +09:00
Gilles Gouaillardet
e1b6ab4f1d ROMIO 3.1.4 refresh: remove old romio 2015-04-30 19:01:23 +09:00
Gilles Gouaillardet
85e77079b4 ROMIO 3.1.4 refresh: use romio from mpich 3.1.4 2015-04-30 19:00:50 +09:00
Gilles Gouaillardet
92f6c7c1e2 ROMIO 3.1.4 refresh: apply post romio-3.1.4 patches 2015-04-30 18:56:53 +09:00
Gilles Gouaillardet
6400bc75ab ROMIO 3.1.4 refresh: patch romio for Open MPI 2015-04-30 18:53:55 +09:00
Gilles Gouaillardet
eacd434a02 ROMIO 3.1.4 refresh: import romio from mpich 3.1.4 tarball 2015-04-30 18:53:03 +09:00
Gilles Gouaillardet
e2e91142d5 ROMIO 3.1.4 refresh: prepare new romio directory 2015-04-30 18:52:22 +09:00
Gilles Gouaillardet
1ee58af8f5 update ROMIO .gitignore 2015-04-30 18:49:53 +09:00
Gilles Gouaillardet
18b75bd40d io/base: check the MCA version matches 2015-04-28 17:48:23 +09:00
Nathan Hjelm
df75d0382f ompi: use C99 subobject naming for component initialization
This commit helps future-proof ompi components by initializing each
component member by name.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-04-18 10:29:58 -06:00
Mangala Jyothi Bhaskar
c4de46e284 Fix number of aggregators used in two phase fcoll 2015-04-16 10:39:10 -05:00
Nathan Hjelm
b68d66bb9b MCA: Add the project/project version to the MCA base component
This commit adds support for project_framework_component_* parameter
matching. This is the first step in allowing the same framework name
in multiple projects. This change also bumps the MCA component version
to 2.1.0.

All master frameworks have been updated to use the new component
versioning macro. An mca.h has been added to each project to add a
project specific versioning macro of the form
PROJECT_MCA_VERSION_2_1_0.

Signed-off-by: Nathan Hjelm <hjelmn@me.com>
2015-03-27 10:59:04 -06:00
Gilles Gouaillardet
f03d7dce17 ompio: fix deallocation sequence
as reported by Coverity with CID 1287034
2015-03-06 14:59:59 +09:00
Gilles Gouaillardet
5b2122381b ompio: fix misc memory leaks
as reported by Coverity with CIDs 72127, 72145, 72146, 72177, 72179,
72186, 731276, 731278, 1269888, 1269890
2015-03-05 16:22:19 +09:00
Howard Pritchard
53fd425a6a romio: patches from Rob Latham for issue #255
Patches supplied by Rob Latham which fix issue #255.

See
http://git.mpich.org/mpich.git/commit/4e80e1d2b9
http://git.mpich.org/mpich.git/commit/5a10283bf7fd

Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
2015-03-02 15:33:49 -08:00
Howard Pritchard
2b5c52fb05 romio: add comment per Rob Latham's request 2015-02-28 11:05:03 -07:00
Gilles Gouaillardet
74a46863ca romio: always set err_ status in MPIO_DATATYPE_ISCOMMITTED
from the ROMIO sources :
/* Any MPI implementation that wishes to follow the thread-safety and
   error reporting features provided by MPICH must implement these
   four functions.  Defining these as empty should not change the behavior
   of correct programs */
the MPIO_DATATYPE_ISCOMMITTED macro now always set err_=0
this is an optimistic approach for Open MPI, but it is likely other upper
layers already checked the datatype was committed.
not setting err_ is incorrect since it can lead to some use of uninitialized
variable.

Fixes open-mpi/ompi#404
2015-02-26 13:39:05 +09:00
Rob Latham
24a6f14257 fix build error if greq extensions not available
Fall back to immediately completing the non-blocking request if we are
unable to use the extended generalized request routines.
2015-02-25 10:52:22 -06:00
Jeff Squyres
657d2cf77b io_ompi: check the correct returned value
This was CID 731273
2015-02-24 15:24:11 -05:00
Howard Pritchard
bf89131f9e add owner files to opa/ompi/orte mca directories
This commit adds an owner file in each of the component directories
for each framework.  This allows for a simple script to parse
the contents of the files and generate, among other things, tables
to be used on the project's wiki page.  Currently there are two
"fields" in the file, an owner and a status.  A tool to parse
the files and generate tables for the wiki page will be added
in a subsequent commit.
2015-02-22 15:10:23 -07:00
Gilles Gouaillardet
b68e1cd579 ompio: fix memory leaks reported by Coverity scan
and silence CID 72195
2015-02-13 13:18:44 +09:00
Jeff Squyres
c368699bde configury: whitespace cleanup
No code changes.
2015-02-09 20:49:04 -08:00
Jeff Squyres
0acdf34044 configury: --enable-dist ceased being useful long ago
The --enable-dist flag is a relic from a prior iteration of the build
system.  It isn't necessary any more.
2015-02-09 20:49:04 -08:00
Edgar Gabriel
62282a0f06 part 1 of cleaning up the info handling in the ompio component. 2015-01-30 16:13:15 -06:00
Edgar Gabriel
2d79963cb3 mark the request as an IO request 2015-01-29 08:41:24 -06:00
Edgar Gabriel
4a5d3e705b remove an unecessary comm_dup in case the internal file_open function is called from a sharedfp component. 2015-01-28 16:13:50 -06:00
Edgar Gabriel
51cee4a083 call the actual backend functions for file_get_position_shared and file_seek_shared. The sharedfp components
do provide the implementations, we just did not invoke them yet.
2015-01-28 13:47:54 -06:00
Gilles Gouaillardet
6d84afc660 romio: update configure for Solaris 10 build 2015-01-14 18:25:55 +09:00
Gilles Gouaillardet
b746a8f584 romio: compile openmpi mpi-io glue 2015-01-08 14:08:46 +09:00
mjbhaskar
2d33b0a745 A fix for memory corruption seen on 32 bit machines 2015-01-07 14:41:44 -06:00
Ralph Castain
e0927895db Grrr...how many files did they forget? 2015-01-06 19:40:18 -08:00
Ralph Castain
84c41429e9 Add missing file 2015-01-06 18:41:11 -08:00
Gilles Gouaillardet
0914de9eae refresh ROMIO based on v3.2a2-84-gef1cf14 2015-01-06 19:43:58 +09:00
Mangala Jyothi Bhaskar
68d78fd718 Aggregator selection logic Part 2 and reorganized Part1 2014-12-16 15:48:40 -06:00
Mangala Jyothi Bhaskar
2bd52cc410 Initialize req variable to fix a warning 2014-12-16 13:24:28 -06:00
Edgar Gabriel
0758d7570e part 1 of the fix to get rid of the missing symbols that prevent the sub-modules to be loaded. 2014-11-29 20:01:36 -06:00
Jeff Squyres
8468424f45 distscript: remove configure.params and autogen.subdirs kruft
Remove configure.params support: configure.params hasn't been used in
years.

Also remove autogen.subdirs support; those should really be handled by
their respective Makefile.am's.
2014-10-02 11:32:54 -07:00
Edgar Gabriel
05c34946f7 implementation of non-blocking read/write operations through aio
functions for the posix module. Som interface changes for the fbtl were
necessary for that.

This commit was SVN r32777.
2014-09-23 21:27:57 +00:00
Mangala Jyothi Bhaskar
dc05b709a7 it is ok to not have a sharedfp component selected, as long as no
sharedfp functionality is being used. Return an error however if no
sharedfp component is selected and the applications calls a
file_read/write_shared function.

This commit was SVN r32718.
2014-09-12 21:15:58 +00:00
Edgar Gabriel
597177cd8b silence a warning regarding the return value of the fbtl's.
This commit was SVN r32717.
2014-09-12 18:01:30 +00:00
Mangala Jyothi Bhaskar
cd78a3a026 Fixed offset data type in communication
This commit was SVN r32710.
2014-09-11 14:51:30 +00:00
Mangala Jyothi Bhaskar
4ff21d6178 Fixed offset data type in communication
This commit was SVN r32709.
2014-09-11 14:51:07 +00:00
Mangala Jyothi Bhaskar
6e5f2c8ae8 Fixed offset data type in communication
This commit was SVN r32708.
2014-09-11 14:50:30 +00:00
Edgar Gabriel
ed02927767 - do not set the contiguous memory option in the collective operations. It
should not be stored on the file handle anyway, since it is not a property of
the file.
- protect a realloc for zero byte scenarios.

This commit was SVN r32678.
2014-09-07 18:09:43 +00:00
Edgar Gabriel
0f59ce6591 use the fbtl return value as originally intended, namely to retrieve the
number of bytes written and read. Status contains now the actual number of
bytes written for individual operations. For collective operations, this is
unfortunately not possible.

This commit was SVN r32674.
2014-09-07 15:14:57 +00:00
Edgar Gabriel
52eac0146d cleanup of the fbtl interfaces: remove the *sorted optimization flag, since it
was not used anyway in the last two years. Simplifies the code significantly.

This commit was SVN r32602.
2014-08-25 18:04:24 +00:00
Ralph Castain
ac0c584eb7 Add missing file
This commit was SVN r32588.
2014-08-23 04:31:35 +00:00
Edgar Gabriel
9987135da0 add initial support for non-blocking read and write operations.
This commit was SVN r32571.
2014-08-22 01:34:19 +00:00
Mangala Jyothi Bhaskar
a5973c3f8c revamp of the aggregator selection logic, part 1.
This commit was SVN r32557.
2014-08-20 19:28:04 +00:00
Edgar Gabriel
fabad95b8e - extend the explicit offset patch to collective explicit offset operations as
well
- minor restructuring to support the shared file pointer operations correctly
  for explicit offsets

This commit was SVN r32538.
2014-08-15 14:03:29 +00:00
Edgar Gabriel
d773dc8aa5 make arbitrary sequences of explicit and implicit offset operations work properly.
This commit was SVN r32537.
2014-08-15 01:49:43 +00:00
Edgar Gabriel
da1b6c2e87 some code reorganization in preparation for non-blocking read and write
operations.

This commit was SVN r32534.
2014-08-14 20:17:58 +00:00
Edgar Gabriel
e401b68ca5 fix the zero byte fileview problem reported by Mohamad on the mailinglist
This commit was SVN r32529.
2014-08-13 23:44:43 +00:00
Ralph Castain
552c9ca5a0 George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT:    Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL

All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies.  This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP.  Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose.  UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs.  A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.

This commit was SVN r32317.
2014-07-26 00:47:28 +00:00