This commit unifies the configury of some verbs-lovin' components.
* Add new configure command line options and deprecate some old ones:
* --with-verbs replaces --with-openib
* --with-verbs-libdir replaces --with-openib-libdir
* If you specify --with-openib[-libdir] without
--with-verbs[-libdir], you'll get a "these options have been
deprecated!" warning, but then they'll act just like
--with-verbs[--libdir].
'''Sidenote:''' Note that we are not renaming any components at this
time, nor are we renaming the top-level OMPI_CHECK_OPENIB m4 macro
(which is pretty strongly tied to the openib BTL and is bastaridzed
by the ofud BTL). Note that there will likely be more changes in
this area coming soon (next week?) when some long-standing changes
move to the SVN trunk: some openib BTL infrastructure will move to
ompi/mca/common, and its configury gets split up / refactored.
We extend our philosophy of other --with-<foo> configure options of
--with-verbs to ''all'' verbs-lovin components:
* If you specify --with-verbs, then all verbs-lovin' components must
configure successfully (or abort). This currently means: OOB ud,
BTL ofud, BTL openib.
* If you specify --with-verbs=DIR, then all verbs-lovin' component
must configure successfully (or abort), and will use DIR to find
verbs headers and libraries.
* If you specify --without-verbs, then all verbs-lovin' components
will be ignored.
This commit also fixes a problem where the --with-openib=DIR form
would not use DIR for ''all'' verbs-lovin' components (I think only
BTL openib and BTL ofud used that DIR). Now all of them do, as does
hwloc (because hwloc has some !OpenFabrics helper functions that
require ibv types from verbs.h).
There's a little new m4 infrastructure worth mentioning:
* If you create a new verbs-lovin' component (i.e., a component that
need verbs), your configure.m4 should
AC_REQUIRE([OPAL_CHECK_VERBS_DIR]).
* You can then use three global shell variables: $opal_want_verbs,
$opal_verbs_dir, $opal_verbs_libdir, which will be set as follows:
* opal_want_verbs will be "yes" and opal_verbs_dir and
opal_verbs_libdir will both be set to directory values, '''OR'''
* opal_want_verbs will be "no" and opal_verbs_dir and
opal_verbs_libdir will both be set empty
This commit was SVN r26640.
This commit is contained in:
Jeff Squyres
parent
8b693b09af
commit
148ae6d6e3
20
README
20
README
@ -711,21 +711,27 @@ NETWORKING SUPPORT / OPTIONS
|
||||
look in <mxm directory>/lib and <mxm directory>/lib64, which covers
|
||||
most cases. This option is only needed for special configurations.
|
||||
|
||||
--with-openib=<directory>
|
||||
Specify the directory where the OpenFabrics (previously known as
|
||||
OpenIB) libraries and header files are located. This option is
|
||||
generally only necessary if the OpenFabrics headers and libraries
|
||||
are not in default compiler/linker search paths.
|
||||
--with-verbs=<directory>
|
||||
Specify the directory where the verbs (also know as OpenFabrics, and
|
||||
previously known as OpenIB) libraries and header files are located.
|
||||
This option is generally only necessary if the verbs headers and
|
||||
libraries are not in default compiler/linker search paths.
|
||||
|
||||
"OpenFabrics" refers to iWARP-, RoCE- (aka "IBoIP"), and
|
||||
InfiniBand-based networks.
|
||||
|
||||
--with-openib-libdir=<directory>
|
||||
Look in directory for the OpenFabrics libraries. By default, Open
|
||||
--with-verbs-libdir=<directory>
|
||||
Look in directory for the verbs libraries. By default, Open
|
||||
MPI will look in <openib directory>/lib and <openib
|
||||
directory>/lib64, which covers most cases. This option is only
|
||||
needed for special configurations.
|
||||
|
||||
--with-openib=<directory>
|
||||
DEPRECATED synonym for --with-verbs.
|
||||
|
||||
--with-openib-libdir=<directory>
|
||||
DEPRECATED synonym for --with-verbs-libdir.
|
||||
|
||||
--with-portals=<directory>
|
||||
Specify the directory where the Portals libraries and header files
|
||||
are located. This option is generally only necessary if the Portals
|
||||
|
||||
164
config/opal_check_verbs.m4
Normal file
164
config/opal_check_verbs.m4
Normal file
@ -0,0 +1,164 @@
|
||||
# -*- shell-script -*-
|
||||
#
|
||||
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
|
||||
# University Research and Technology
|
||||
# Corporation. All rights reserved.
|
||||
# Copyright (c) 2004-2005 The University of Tennessee and The University
|
||||
# of Tennessee Research Foundation. All rights
|
||||
# reserved.
|
||||
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
||||
# University of Stuttgart. All rights reserved.
|
||||
# Copyright (c) 2004-2005 The Regents of the University of California.
|
||||
# All rights reserved.
|
||||
# Copyright (c) 2006-2012 Cisco Systems, Inc. All rights reserved.
|
||||
# Copyright (c) 2006-2011 Los Alamos National Security, LLC. All rights
|
||||
# reserved.
|
||||
# Copyright (c) 2006-2009 Mellanox Technologies. All rights reserved.
|
||||
# Copyright (c) 2010-2012 Oracle and/or its affiliates. All rights reserved.
|
||||
# $COPYRIGHT$
|
||||
#
|
||||
# Additional copyrights may follow
|
||||
#
|
||||
# $HEADER$
|
||||
#
|
||||
|
||||
|
||||
# Internal helper macro to look for the verbs libdir
|
||||
# --------------------------------------------------------
|
||||
AC_DEFUN([_OPAL_CHECK_VERBS_LIBDIR],[
|
||||
AS_IF([test -d "$1"],
|
||||
[AS_IF([test "x`ls $1/libibverbs.* 2> /dev/null`" != "x"],
|
||||
[opal_verbs_libdir="$1"])
|
||||
])
|
||||
])
|
||||
|
||||
# Internal helper macro to look for the verbs dir
|
||||
# --------------------------------------------------------
|
||||
AC_DEFUN([_OPAL_CHECK_VERBS_DIR],[
|
||||
AS_IF([test -d "$1"],
|
||||
[AS_IF([test -f "$1/include/infiniband/verbs.h"],
|
||||
[opal_verbs_dir="$1"])
|
||||
])
|
||||
])
|
||||
|
||||
# OPAL_CHECK_VERBS_DIR
|
||||
# --------------------------------------------------------
|
||||
# Add --with-verbs options, and if directories are specified,
|
||||
# sanity check them.
|
||||
#
|
||||
# At the end of this macro:
|
||||
#
|
||||
# 1. $opal_want_verbs will be set to "yes" and both of
|
||||
# $opal_verbs_dir and $opal_verbs_libdir will be set
|
||||
#
|
||||
# or
|
||||
#
|
||||
# 2. $opal_want_verbs will be set to "no" and both of
|
||||
# $opal_verbs_dir and $opal_verbs_libdir will be empty
|
||||
#
|
||||
AC_DEFUN([OPAL_CHECK_VERBS_DIR],[
|
||||
|
||||
# Add --with options
|
||||
AC_ARG_WITH([verbs],
|
||||
[AC_HELP_STRING([--with-verbs(=DIR)],
|
||||
[Build verbs support, optionally adding DIR/include, DIR/lib, and DIR/lib64 to the search path for headers and libraries])])
|
||||
AC_ARG_WITH([verbs-libdir],
|
||||
[AC_HELP_STRING([--with-verbs-libdir=DIR],
|
||||
[Search for verbs libraries in DIR])])
|
||||
|
||||
# Add deprecated --with options
|
||||
# JMS REMOVE THESE IN v1.9
|
||||
AC_ARG_WITH([openib],
|
||||
[AC_HELP_STRING([--with-openib(=DIR)],
|
||||
[DEPRECATED synonym for --with-verbs])])
|
||||
AC_ARG_WITH([openib-libdir],
|
||||
[AC_HELP_STRING([--with-openib-libdir=DIR],
|
||||
[DEPRECATED synonym for --with-verbs-libdir])])
|
||||
|
||||
# Did we specify both --with-openib* and --with-verbs*?
|
||||
# JMS REMOVE THESE IN v1.9
|
||||
AS_IF([test -n "$with_openib" -a -n "$with_verbs"],
|
||||
[AC_MSG_WARN([Both --with-verbs and --with-openib specified.])
|
||||
AC_MSG_WARN([--with-openib value ignored.])
|
||||
with_openib=
|
||||
])
|
||||
AS_IF([test -n "$with_openib_libdir" -a -n "$with_verbs_libdir"],
|
||||
[AC_MSG_WARN([Both --with-verbs-libdir and --with-openib-libdir specified.])
|
||||
AC_MSG_WARN([--with-openib-libdir value ignored.])
|
||||
with_openib_libdir=
|
||||
])
|
||||
|
||||
# Did we specify --with-openib* and not --with-verbs*?
|
||||
# JMS REMOVE THESE IN v1.9
|
||||
AS_IF([test -n "$with_openib" -a -z "$with_verbs"],
|
||||
[AC_MSG_WARN([**************************************************])
|
||||
AC_MSG_WARN([The use of --with-openib is deprecated.])
|
||||
AC_MSG_WARN([You should use --with-verbs instead.])
|
||||
AC_MSG_WARN([Proceeding with the assumption you meant])
|
||||
AC_MSG_WARN([--with-verbs=$with_openib])
|
||||
AC_MSG_WARN([(sleeping to let you read this message)])
|
||||
AC_MSG_WARN([**************************************************])
|
||||
sleep 10
|
||||
with_verbs=$with_openib
|
||||
with_openib=
|
||||
])
|
||||
AS_IF([test -n "$with_openib_libdir" -a -z "$with_verbs_libdir"],
|
||||
[AC_MSG_WARN([**************************************************])
|
||||
AC_MSG_WARN([The use of --with-openib-libdir is deprecated.])
|
||||
AC_MSG_WARN([You should use --with-verbs-libdir instead.])
|
||||
AC_MSG_WARN([Proceeding with the assumption you meant])
|
||||
AC_MSG_WARN([--with-verbs-libdir=$with_openib_libdir])
|
||||
AC_MSG_WARN([(sleeping to let you read this message)])
|
||||
AC_MSG_WARN([**************************************************])
|
||||
sleep 10
|
||||
with_verbs_libdir=$with_openib_libdir
|
||||
with_openib_libdir=
|
||||
])
|
||||
|
||||
# Sanity check the --with values
|
||||
OMPI_CHECK_WITHDIR([verbs], [$with_verbs],
|
||||
[include/infiniband/verbs.h])
|
||||
OMPI_CHECK_WITHDIR([verbs-libdir], [$with_verbs_libdir],
|
||||
[libibverbs.*])
|
||||
|
||||
# Set standardized shell variables for OFED lovin' components to
|
||||
# use. Either both of $opal_verbs_dir and
|
||||
# $verbs_libdir will be set, or neither will be set.
|
||||
opal_want_verbs=no
|
||||
opal_verbs_dir=
|
||||
AS_IF([test -n "$with_verbs" -a "$with_verbs" != "yes" -a "$with_verbs" != "no"],
|
||||
[opal_want_verbs=yes
|
||||
opal_verbs_dir=$with_verbs
|
||||
])
|
||||
|
||||
opal_verbs_libdir=
|
||||
AS_IF([test -n "$with_verbs_libdir" -a "$with_verbs_libdir" != "yes" -a "$with_verbs_libdir" != "no"],
|
||||
[opal_want_verbs=yes
|
||||
opal_verbs_libdir=$with_verbs_libdir
|
||||
])
|
||||
|
||||
# If the top dir was specified but the libdir was not, look for
|
||||
# it. Note that if the user needs a specific libdir (i.e., if our
|
||||
# hueristic ordering below is not sufficient), they need to
|
||||
# specify it.
|
||||
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
|
||||
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib"])])
|
||||
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
|
||||
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib64"])])
|
||||
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
|
||||
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib32"])])
|
||||
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
|
||||
[AC_MSG_WARN([Could not find libibverbs in the usual locations under $opal_verbs_dir])
|
||||
AC_MSG_ERROR([Cannot continue])
|
||||
])
|
||||
|
||||
# If the libdir was specified, but the top dir was not, look for
|
||||
# it. Note that if the user needs a specific top dir (i.e., if
|
||||
# our hueristic below is not sufficient), they need to specify it.
|
||||
AS_IF([test -z "$opal_verbs" -a -n "$opal_verbs_libdir"],
|
||||
[_OPAL_CHECK_VERBS_DIR([`dirname "$opal_verbs_libdir"`])])
|
||||
AS_IF([test -z "$opal_verbs_dir" -a -n "$opal_verbs_libdir"],
|
||||
[AC_MSG_WARN([Could not find verbs.h in the usual locations under $opal_verbs_dir])
|
||||
AC_MSG_ERROR([Cannot continue])
|
||||
])
|
||||
])
|
||||
@ -10,7 +10,7 @@
|
||||
# University of Stuttgart. All rights reserved.
|
||||
# Copyright (c) 2004-2005 The Regents of the University of California.
|
||||
# All rights reserved.
|
||||
# Copyright (c) 2006-2010 Cisco Systems, Inc. All rights reserved.
|
||||
# Copyright (c) 2006-2012 Cisco Systems, Inc. All rights reserved.
|
||||
# Copyright (c) 2006-2011 Los Alamos National Security, LLC. All rights
|
||||
# reserved.
|
||||
# Copyright (c) 2006-2009 Mellanox Technologies. All rights reserved.
|
||||
@ -31,25 +31,17 @@
|
||||
AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
OPAL_VAR_SCOPE_PUSH([$1_msg])
|
||||
|
||||
#
|
||||
# Openfabrics support
|
||||
#
|
||||
AC_ARG_WITH([openib],
|
||||
[AC_HELP_STRING([--with-openib(=DIR)],
|
||||
[Build OpenFabrics support, optionally adding DIR/include, DIR/lib, and DIR/lib64 to the search path for headers and libraries])])
|
||||
OMPI_CHECK_WITHDIR([openib], [$with_openib], [include/infiniband/verbs.h])
|
||||
AC_ARG_WITH([openib-libdir],
|
||||
[AC_HELP_STRING([--with-openib-libdir=DIR],
|
||||
[Search for OpenFabrics libraries in DIR])])
|
||||
OMPI_CHECK_WITHDIR([openib-libdir], [$with_openib_libdir], [libibverbs.*])
|
||||
# Setup the --with switches to allow users to specify where
|
||||
# verbs stuff lives.
|
||||
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
|
||||
|
||||
#
|
||||
# Add padding to OpenIB header
|
||||
#
|
||||
AC_ARG_ENABLE([openib-control-hdr-padding],
|
||||
[AC_HELP_STRING([--enable-openib-control-hdr-padding],
|
||||
[Add padding bytes to the openib control header (default:disabled)])])
|
||||
AC_MSG_CHECKING([if want to add padding to the openib control header])
|
||||
[Add padding bytes to the openib BTL control header (default:disabled)])])
|
||||
AC_MSG_CHECKING([if want to add padding to the openib BTL control header])
|
||||
if test "$enable_openib_control_hdr_padding" = "yes"; then
|
||||
AC_MSG_RESULT([yes])
|
||||
ompi_openib_pad_hdr=1
|
||||
@ -75,14 +67,14 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
esac
|
||||
fi
|
||||
AC_DEFINE_UNQUOTED([OMPI_OPENIB_PAD_HDR], [$ompi_openib_pad_hdr],
|
||||
[Add padding bytes to the openib control header])
|
||||
[Add padding bytes to the openib BTL control header])
|
||||
|
||||
#
|
||||
# ConnectX XRC support
|
||||
#
|
||||
AC_ARG_ENABLE([openib-connectx-xrc],
|
||||
[AC_HELP_STRING([--enable-openib-connectx-xrc],
|
||||
[Enable ConnectX XRC support. If you do not have InfiniBand ConnectX adapters, you may disable the ConnectX XRC support. If you do not know which InfiniBand adapter is installed on your cluster, leave this option enabled (default: enabled)])],
|
||||
[Enable ConnectX XRC support in the openib BTL. If you do not have InfiniBand ConnectX adapters, you may disable the ConnectX XRC support. If you do not know which InfiniBand adapter is installed on your cluster, leave this option enabled (default: enabled)])],
|
||||
[enable_connectx_xrc="$enableval"], [enable_connectx_xrc="yes"])
|
||||
|
||||
#
|
||||
@ -100,11 +92,7 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
[AC_HELP_STRING([--enable-openib-rdmacm],
|
||||
[Enable Open Fabrics RDMACM support in openib BTL (default: enabled)])])
|
||||
|
||||
AS_IF([test ! -z "$with_openib" -a "$with_openib" != "yes"],
|
||||
[ompi_check_openib_dir="$with_openib"])
|
||||
AS_IF([test ! -z "$with_openib_libdir" -a "$with_openib_libdir" != "yes"],
|
||||
[ompi_check_openib_libdir="$with_openib_libdir"])
|
||||
AS_IF([test "$with_openib" = "no"],
|
||||
AS_IF([test "$opal_want_verbs" = "no"],
|
||||
[ompi_check_openib_happy="no"],
|
||||
[ompi_check_openib_happy="yes"])
|
||||
|
||||
@ -140,8 +128,8 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
[ibverbs],
|
||||
[ibv_open_device],
|
||||
[],
|
||||
[$ompi_check_openib_dir],
|
||||
[$ompi_check_openib_libdir],
|
||||
[$opal_verbs_dir],
|
||||
[$opal_verbs_libdir],
|
||||
[ompi_check_openib_happy="yes"],
|
||||
[ompi_check_openib_happy="no"])])
|
||||
|
||||
@ -321,9 +309,9 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
AC_MSG_RESULT([no])
|
||||
fi
|
||||
|
||||
AS_IF([test -z "$ompi_check_openib_dir"],
|
||||
AS_IF([test -z "$opal_verbs_dir"],
|
||||
[openib_include_dir="/usr/include"],
|
||||
[openib_include_dir="$ompi_check_openib_dir/include"])
|
||||
[openib_include_dir="$opal_verbs_dir/include"])
|
||||
|
||||
CPPFLAGS="$ompi_check_openib_$1_save_CPPFLAGS -I$openib_include_dir/infiniband"
|
||||
LDFLAGS="$ompi_check_openib_$1_save_LDFLAGS"
|
||||
@ -331,12 +319,11 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
|
||||
|
||||
AS_IF([test "$ompi_check_openib_happy" = "yes"],
|
||||
[$2],
|
||||
[AS_IF([test ! -z "$with_openib" -a "$with_openib" != "no"],
|
||||
[AC_MSG_WARN([OpenFabrics support requested (via --with-openib) but not found.])
|
||||
[AS_IF([test "$opal_want_verbs" = "yes"],
|
||||
[AC_MSG_WARN([Verbs support requested (via --with-verbs) but not found.])
|
||||
AC_MSG_WARN([If you are using libibverbs v1.0 (i.e., OFED v1.0 or v1.1), you *MUST* have both the libsysfs headers and libraries installed. Later versions of libibverbs do not require libsysfs.])
|
||||
AC_MSG_ERROR([Aborting.])])
|
||||
$3])
|
||||
|
||||
OPAL_VAR_SCOPE_POP
|
||||
])
|
||||
|
||||
|
||||
@ -40,6 +40,9 @@ AC_DEFUN([MCA_opal_hwloc_hwloc142_POST_CONFIG],[
|
||||
# MCA_hwloc_hwloc142_CONFIG([action-if-found], [action-if-not-found])
|
||||
# --------------------------------------------------------------------
|
||||
AC_DEFUN([MCA_opal_hwloc_hwloc142_CONFIG],[
|
||||
# Hwloc needs to know if we have Verbs support
|
||||
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
|
||||
|
||||
AC_CONFIG_FILES([opal/mca/hwloc/hwloc142/Makefile])
|
||||
|
||||
OPAL_VAR_SCOPE_PUSH([HWLOC_VERSION opal_hwloc_hwloc142_save_CPPFLAGS opal_hwloc_hwloc142_save_LDFLAGS opal_hwloc_hwloc142_save_LIBS opal_hwloc_hwloc142_save_cairo opal_hwloc_hwloc142_save_xml opal_hwloc_hwloc142_basedir opal_hwloc_hwloc142_file opal_hwloc_hwloc142_save_cflags])
|
||||
@ -110,7 +113,11 @@ AC_DEFUN([MCA_opal_hwloc_hwloc142_CONFIG],[
|
||||
[Version of hwloc])
|
||||
|
||||
# Do we have verbs support?
|
||||
CPPFLAGS_save=$CPPFLAGS
|
||||
AS_IF([test "$opal_want_verbs" = "yes"],
|
||||
[CPPFLAGS="-I$opal_verbs_dir/include $CPPFLAGS"])
|
||||
AC_CHECK_HEADERS([infiniband/verbs.h])
|
||||
CPPFLAGS=$CPPFLAGS_save
|
||||
|
||||
# Set these variables so that the framework m4 knows
|
||||
# what file to include in opal/mca/hwloc/hwloc.h
|
||||
|
||||
@ -10,6 +10,7 @@
|
||||
# University of Stuttgart. All rights reserved.
|
||||
# Copyright (c) 2004-2005 The Regents of the University of California.
|
||||
# All rights reserved.
|
||||
# Copyright (c) 2012 Cisco Systems, Inc. All rights reserved.
|
||||
# $COPYRIGHT$
|
||||
#
|
||||
# Additional copyrights may follow
|
||||
@ -20,6 +21,9 @@
|
||||
# MCA_oob_ud_CONFIG([action-if-found], [action-if-not-found])
|
||||
# -----------------------------------------------------------
|
||||
AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
|
||||
# We need to know if we have verbs support
|
||||
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
|
||||
|
||||
AC_CONFIG_FILES([orte/mca/oob/ud/Makefile])
|
||||
|
||||
# JMS Still have problems with AC_ARG ENABLE not yet having been
|
||||
@ -34,8 +38,8 @@ AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
|
||||
[ibverbs],
|
||||
[ibv_open_device],
|
||||
[],
|
||||
[$ompi_check_openib_dir],
|
||||
[$ompi_check_openib_libdir],
|
||||
[$opal_verbs_dir],
|
||||
[$opal_verbs_libdir],
|
||||
[orte_oob_ud_check_happy=yes],
|
||||
[orte_oob_ud_check_happy=no])
|
||||
|
||||
@ -45,7 +49,11 @@ AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
|
||||
|
||||
AS_IF([test "$orte_oob_ud_check_happy" = "yes"],
|
||||
[$1],
|
||||
[$2])
|
||||
[AS_IF([test "$opal_want_verbs" = "yes"],
|
||||
[AC_MSG_WARN([--with-verbs specified, but cannot build this component])
|
||||
AC_MSG_ERROR([Cannot continue])
|
||||
])
|
||||
$2])
|
||||
|
||||
# substitute in the things needed to build this component
|
||||
AC_SUBST([orte_oob_ud_CFLAGS])
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user