1
1

119 Коммитов

Автор SHA1 Сообщение Дата
Edgar Gabriel
b253d4e887 fix CID 1349739, CID 1349738, CID 1349736 and (probably) CID 1349740 (not entirely sure about the last one, since I don't understand why block[i] is a problem but max_len[i] allocated and treated exactly the same way 1 line later is not). 2016-01-21 08:32:23 -06:00
Edgar Gabriel
a9ca37059a improve the communicaton abstraction. This commit also allows all aggregators to work simultaniously, instead of the slightly staggered way of the previous version. 2016-01-17 09:48:49 -06:00
Edgar Gabriel
26c57ef374 separate the size of the buffer used for the shuffle step and the size of the buffer used for a pwritev operation. 2016-01-17 09:48:49 -06:00
Edgar Gabriel
39d5c8c281 further bug fixes silencing a compiler warning and fixing a memory overrun 2016-01-17 09:48:49 -06:00
Edgar Gabriel
2bcae84e11 further debugging 2016-01-17 09:48:49 -06:00
Edgar Gabriel
2bdd6ba17a correctly free some buffers, and ensure that lustre_stripe_size and stripe_count are always read from the file system. 2016-01-17 09:48:49 -06:00
Edgar Gabriel
d282e94b67 add the new dynamic_gen2 component, designed to coexist for now with the original dynamic component 2016-01-17 09:48:49 -06:00
Gilles Gouaillardet
bfe8e03d9d fcoll/two_phase: use ompi_mpi_abort instead of PMPI_Abort
Thanks Jeff for the review
2015-12-07 11:34:36 +09:00
Gilles Gouaillardet
002c7b8b3a fcoll/two_phase: use PMPI_* insted of MPI_* 2015-11-20 13:46:19 +09:00
Nathan Hjelm
5122327727 fcoll/two_phase: fix new coverity errors
Fix CID 1325467: use after free

Remove extra free of aggregator_list.

Fix CID 1325466: resource leak

Fix typo in prior coverity fix.

Signed-off-by: Nathan Hjelm <hjelmn@me.com>
2015-10-02 21:38:31 -06:00
Nathan Hjelm
95b95e19af fcoll/dynamic: fix coverity errors
Fixes CID 72320: Explicit NULL dereferenced

On error it is possible that the blocklen_per_process array is
NULL. Change the NULL check before the free to check for non-NULL on
the array not the array element. Also clean up allocation of this
array to use calloc instead of malloc + setting each element to NULL.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-10-01 14:38:09 -06:00
Nathan Hjelm
09df7aa205 fcoll/two_phase: fix coverity errors
Fixes CIDs 72300, 72344, 1196764-1196768, 72300: Resource leaks

Mulitple allocated arrays are going out of scope at the end of
mca_fcoll_two_phase_file_write_all. Free these arrays. Also removed
the extraneous NULL checks since free (NULL) is safe in C.

Change returns to goto exit where the allocated resources are freed.

Fixes CIDs 72285-72292, 72297, 72298: Resource leaks

Change all appropriate return statements to goto exit to ensure that
all resources are freed. Also removed the NULL checks since free
(NULL) is safe in C.

Fixes CIDs 72295, 72296: Resource leaks

Moved free of requests and recv_types to after exit label. This will
ensure these are freed on error.

Also added a loop and statement to free send_buf which is going out of
scope at the end of the function.

Fixes CIDs 72336-72240, 735197, 735198: Resource leaks

Moved the exit label before to before the resources are released and
changed all appropriate return statements to goto exit. Also removed
extraneous NULL checks because free (NULL) is safe in C.

Fixes CIDs 72341, 72343, 1196805-1196809: Resource leaks

Free all resources after exit label and change return statements to
goto exit to ensure all resources are freed on error.

Fixes CID 1269973: Unused value

Check return code of ompi_request_wait_all. If it fails jump to the
exit.

Fixes CID 714119: Dereference before NULL check

Wrong value checked in conditional.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-10-01 14:38:09 -06:00
Edgar Gabriel
01fcfb08fe do not set the contigous flag in two_phase_file_read_all. This optimization
needs some more debugging for the two_phase component, and is disabled
for two_phase_file_write_all as well.
2015-09-18 09:30:50 -05:00
Gilles Gouaillardet
fe351f6801 io: do not cast way the const modifier when this is not necessary
update the io framework and mpi c bindings
2015-09-09 09:18:58 +09:00
Jeff Squyres
bc9e5652ff whitespace: purge whitespace at end of lines
Generated by running "./contrib/whitespace-purge.sh".
2015-09-08 09:47:17 -07:00
Edgar Gabriel
c83e6ad0c8 fix coverty warnings 1322865 and 72136 2015-09-08 09:15:57 -05:00
Edgar Gabriel
c9710660af Merge pull request #863 from edgargabriel/topic/fcoll-static-cleanup
Topic/fcoll static cleanup
2015-09-03 11:21:02 -05:00
Edgar Gabriel
a96a15a83c re-enable the contiguous buffer optimization similarly to the dynamic component. Passes all hdf5testsi and our own test suite.
Please enter the commit message for your changes. Lines starting
2015-09-03 10:13:03 -05:00
Edgar Gabriel
8007effc93 code cleanup for static component, similarly to the dynamic one 2015-09-03 10:12:45 -05:00
Edgar Gabriel
ac3a01c39c Silence coverty warnings 1321702, 1321701, 1321700, 72331, 72330, 72327, 72326, 72325, 2015-09-03 09:10:25 -05:00
Edgar Gabriel
82efc23e8d iclean up indenting and tabs/space of fcoll_static_file_read/write_all 2015-09-01 09:39:33 -05:00
Edgar Gabriel
a1778406d6 Re-enable the contiguous buffer optimization to the read_all and the write_all routines.
After long debugging, I found last week the reason this optimization originally broke
some hdf5 tests. We now pass the hdf5 test suite with the optimization being actively used.
2015-09-01 09:29:07 -05:00
Edgar Gabriel
c2c44b11dc Code cleanup for dynamic read_all and write_all
Specifically:
 - reduce the number of realloc's and malloc's by moving
   some arrays out of the cycle loop, if we know that there
   size is not changing
 - store the rank of the aggregator in a separate variable to avoid
   continuous dereferencing
 - change the wait_all logic in write_all to use a fix number of requests
   (even if they are MPI_REQUEST_NULL)
 - fix the timing to considere the two initial allgather and the one
   allgatherv operation to be a part of it
 - add more comments.
2015-09-01 09:29:07 -05:00
Edgar Gabriel
cf1e4e0d35 step 0: clean up indenting and space vs. tabs 2015-09-01 09:29:07 -05:00
Edgar Gabriel
6f2e8d2073 last nights coverty fix introduced a new coverty complain. This commit tries to fix the new complain by coverty. 2015-08-25 08:46:38 -05:00
Edgar Gabriel
db2d37ad93 correctly free some arrays in case of an error. This fixes a whole bunch of coverty warnings. 2015-08-24 14:13:37 -05:00
Edgar Gabriel
072b18e197 Code cleanup for the time breakdown feature in ompio/fcoll
- make the internal structure follow the Open MPI naming convention
 - provide a single flag/macro which controls the compilation/utilization of this
   feature, to avoid that somebody using this has to modify every single
   fcoll component. A configure option could be added later if desired.
2015-08-14 08:53:04 -05:00
Edgar Gabriel
16d4171f6b the individual component should call internal ompio functions directly. The reason is that otherwise
the redirection to the ompi_file_t structure (and back to the ompio internal structure) is ambiguise and wrong
for the shared file pointer scenario.
2015-08-05 14:31:11 -05:00
Edgar Gabriel
93a303ba89 Performance tuning: make sure the individual component is selected for 1 and 2 process communicators (important for some benchmarks) 2015-07-30 17:31:16 -05:00
Ralph Castain
869041f770 Purge whitespace from the repo 2015-06-23 20:59:57 -07:00
Edgar Gabriel
3b11a8b61c making the current work compile. 2015-06-18 05:56:51 -05:00
Edgar Gabriel
cc219281ba checkpoint of the current work, since I need to resync wioth master to fix the compilation problems 2015-06-18 05:20:07 -05:00
Edgar Gabriel
100515e321 remove split collective interfaces from fcoll and their fake implemenations. Not required anymore 2015-06-18 05:20:07 -05:00
Edgar Gabriel
19cac73a9b first part of the changes trequired to support non-blocking colelctive io operations 2015-06-18 05:20:07 -05:00
Gilles Gouaillardet
0f17cdfc57 fcoll: fix misc memory leaks
as reported by Coverity with CIDs 72293,72294 and 1269894
2015-06-17 11:17:52 +09:00
Nathan Hjelm
033894b493 Merge pull request #541 from hjelmn/c99_components
C99 component initialization
2015-04-20 10:45:39 -06:00
Howard Pritchard
3339274136 Merge pull request #542 from hppritcha/topic/coverity_714118
fcoll/two_phase: coverity fix
2015-04-20 05:42:12 -06:00
Howard Pritchard
de215addc6 fcoll/two_phase: coverity fix
fix CID 714118

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-04-18 14:34:48 -06:00
Nathan Hjelm
df75d0382f ompi: use C99 subobject naming for component initialization
This commit helps future-proof ompi components by initializing each
component member by name.

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-04-18 10:29:58 -06:00
Mangala Jyothi Bhaskar
c4de46e284 Fix number of aggregators used in two phase fcoll 2015-04-16 10:39:10 -05:00
Nathan Hjelm
3436f2917d Merge pull request #449 from hjelmn/mca_base_update
mca/base update
2015-04-16 08:41:48 -06:00
Howard Pritchard
05324e32ff fcoll/static: coverity fixes
Fix CIDs 72138, 72139, 72143

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-04-02 14:51:44 -06:00
Nathan Hjelm
b68d66bb9b MCA: Add the project/project version to the MCA base component
This commit adds support for project_framework_component_* parameter
matching. This is the first step in allowing the same framework name
in multiple projects. This change also bumps the MCA component version
to 2.1.0.

All master frameworks have been updated to use the new component
versioning macro. An mca.h has been added to each project to add a
project specific versioning macro of the form
PROJECT_MCA_VERSION_2_1_0.

Signed-off-by: Nathan Hjelm <hjelmn@me.com>
2015-03-27 10:59:04 -06:00
Howard Pritchard
66fee3bd18 fcoll/static: minor fix for coverity
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-11 09:11:49 -06:00
Howard Pritchard
fba88360a8 fcoll/dynamic: more coverity fixes
Okay coverity seems to get one stuck in a loop where
by fixing one set of resource allocation problems, it
starts finding more.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-09 15:01:05 -07:00
Howard Pritchard
2d61a652c8 fcoll/dynamic: coverity fixes
okay, hopefully really fix CIDS 72325-72328, and 72330-72332.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-09 13:53:52 -07:00
Howard Pritchard
209f002200 fcoll/static: fix an errant free
Got to excited about coverity and ended up generating
a new coverity error.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-06 13:12:53 -07:00
Howard Pritchard
4f4b99bbac fcoll/dynamic,static: coverity fixes
Fix some theoretical memory leaks reported by coverity.

Fixes CIDS 72332, 72328, 72332, 72138, 72139, 72140, 72364, 72365-72370
           72372-72374, 741354, 72143, 72375-83, 1027140, 1269903

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
2015-03-06 11:05:23 -07:00
Nathan Hjelm
5f1254d710 Update code base to use the new opal_free_list_t
Use of the old ompi_free_list_t and ompi_free_list_item_t is
deprecated. These classes will be removed in a future commit.

This commit updates the entire code base to use opal_free_list_t and
opal_free_list_item_t.

Notes:

OMPI_FREE_LIST_*_MT -> opal_free_list_* (uses opal_using_threads ())

Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
2015-02-24 10:05:45 -07:00
Jeff Squyres
c7d8563d8d fcoll_two_phase: fix trivial compiler warning 2015-02-23 08:58:43 -08:00