Also include two other minor changes:
1. More C99-style member initialization in the component struct
1. Fix the BTL module member initialization to not be redundant
Due to the nature of the cache architecture on power,
we don't export coherency_line_size for L2 in sysfs.
If we are unable to get the L2 cache line size, try L1.
See open-mpi/ompi#383 for more information.
In order to have an effect, ibv_fork_init should be called in the
beginning of the verbs initialization flow - before the calls to the
ibv_create_qp and ibv_create_cq verbs.
These functions are called from the oob/ud code and by the time the
other verbs components (btl openib, pml yalla, ...) call ibv_fork_init,
it's too late. This commit forces the call to ibv_fork_init (if it's
requested) right at the beginning of all the components that are using
verbs.
(ibv_fork_init() can be safely called multiple times)
This commit also removes the btl_openib_want_fork_support mca parameter
and adds a new mca parameter instead - opal_verbs_want_fork_support.
Through this new parameter, fork support may be requested for ALL
components.
The default value for this parameter is set to 1.
Before this commit the btl_openib_want_fork_support parameter didn't
provide fork support for the openib btl if its value was set to 1.
(because when openib called ibv_fork_init, it was already after the
calls to ibv_create_* in oob/ud and thereofre it failed).
Use of the old ompi_free_list_t and ompi_free_list_item_t is
deprecated. These classes will be removed in a future commit.
This commit updates the entire code base to use opal_free_list_t and
opal_free_list_item_t.
Notes:
OMPI_FREE_LIST_*_MT -> opal_free_list_* (uses opal_using_threads ())
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
Please verify your components have been updated correctly. Keep in
mind that in terms of threading:
OPAL_FREE_LIST_GET -> opal_free_list_get_st
OPAL_FREE_LIST_RETURN -> opal_free_list_return_st
I used the opal_using_threads() variant anytime it appeared multiple
threads could be operating on the free list. If this is not the case
update to _st. If multiple threads are always in use change to _mt.
Historically these two lists were different due to ompi_free_list_t
dependencies in ompi (mpool). Those dependencies have since been moved
to opal so it is safe to (finally) combine them. The combined free
list comes in three flavors:
- Single-threaded. Only to be used when it is guaranteed that no
concurrent access will be made to the free list. Single-threaded
functions are suffixed with _st.
- Mutli-threaded. To be used when the free list may be accessed by
multiple threads despite the setting of opal_using_threads.
Multi-threaded functins are suffixed with _mt.
- Conditionally multi-threaded. Common use case. These functions are
thread-safe if opal_using_threads is set to true.
Compatibility functions for the ompi_free_list_t and the old accessor
functions (OPAL_FREE_LIST_*) are available while the code base is
transitioned to the new class/functions.
Signed-off-by: Nathan Hjelm <hjelmn@lanl.gov>
This commit adds an owner file in each of the component directories
for each framework. This allows for a simple script to parse
the contents of the files and generate, among other things, tables
to be used on the project's wiki page. Currently there are two
"fields" in the file, an owner and a status. A tool to parse
the files and generate tables for the wiki page will be added
in a subsequent commit.
Some BTLs do not require local registration for some rdma
transactions. For example: inline put on openib, fma put on ugni. This
commit adds code to expose the local registration thresholds to BTL
users. Optimized code can take advantage of this information to
improve rdma performance.
@ggouaillardet identified that HAVE_ALIAS_ATTRIBUTE was not properly
being defined in the embedded libfabric. This is because the
embedded configury missed the test for it (i.e., the real configure.ac
for libfabric always defines HAVE_ALIAS_ATTRIBUTE to 0 or 1 -- we
didn't emulate that properly here in libfabric's configure.m4).
Also, fix some grammar and properly escape another AC_MSG_CHECKING
message in libfabric's configure.m4.
When configured --with-devel-headers, there's now 2 "osd.h" header
files in libfabric (in different dirs). Automake's "install" target
didn't like this, and errored out.
Since embedding libfabric is a temporary measure, just avoid the
problem by not installing any libfabric headers.
