Edgar Gabriel
19fe5cac50
io/ompio: next step in code-reorganization
...
- move the sort_iovec operations to fcoll/base
- move set_view_internal to common/ompio
- move set_file_default to common/ompio
- remove io_ompio_sort, not used anymore.
2016-08-02 09:18:29 -05:00
Edgar Gabriel
ccf76b7791
moving the internal read/write functions to common/ompio
...
and update all fs/fcoll/sharedfp components to use these functions.
2016-07-21 13:08:32 -05:00
Edgar Gabriel
39ae93b87b
modify the fcoll components to use the common/ompio print queues
2016-07-21 13:08:32 -05:00
Edgar Gabriel
195ec89732
fcoll/base: mv coll_array functionis to fcoll base
...
the coll_array functions are truly only used by the fcoll modules, so move
them to fcoll/base. There is currently one exception to that rule (number of aggreagtors
logic), but that function will be moved in a long term also to fcoll/base.
2016-07-14 08:41:14 -05:00
Edgar Gabriel
92d1b99468
optimize the shuffle step:
...
1. use communicator collectives if possible for performance reasons
2. combined multiple allgathers into a single one
2016-02-19 11:04:04 -06:00
Edgar Gabriel
e63836c653
clean up the mca parameter handling of the component. Add new parameters for number of sub groups and write chunk size. This will allow to perform a systematic parameter study.
2016-02-19 10:15:28 -06:00
Edgar Gabriel
268d525053
change the tag to be a positive value. handle 0-byte situations correctly.
2016-02-19 08:28:50 -06:00
Edgar Gabriel
ad79012059
first cut on the version which overlaps the communication/computation of 2 iterations.
2016-02-19 08:28:50 -06:00
Edgar Gabriel
b253d4e887
fix CID 1349739, CID 1349738, CID 1349736 and (probably) CID 1349740 (not entirely sure about the last one, since I don't understand why block[i] is a problem but max_len[i] allocated and treated exactly the same way 1 line later is not).
2016-01-21 08:32:23 -06:00
Edgar Gabriel
a9ca37059a
improve the communicaton abstraction. This commit also allows all aggregators to work simultaniously, instead of the slightly staggered way of the previous version.
2016-01-17 09:48:49 -06:00
Edgar Gabriel
26c57ef374
separate the size of the buffer used for the shuffle step and the size of the buffer used for a pwritev operation.
2016-01-17 09:48:49 -06:00
Edgar Gabriel
39d5c8c281
further bug fixes silencing a compiler warning and fixing a memory overrun
2016-01-17 09:48:49 -06:00
Edgar Gabriel
2bcae84e11
further debugging
2016-01-17 09:48:49 -06:00
Edgar Gabriel
2bdd6ba17a
correctly free some buffers, and ensure that lustre_stripe_size and stripe_count are always read from the file system.
2016-01-17 09:48:49 -06:00
Edgar Gabriel
d282e94b67
add the new dynamic_gen2 component, designed to coexist for now with the original dynamic component
2016-01-17 09:48:49 -06:00
