1
1

This commit unifies the configury of some verbs-lovin' components.

* Add new configure command line options and deprecate some old ones:
   * --with-verbs replaces --with-openib
   * --with-verbs-libdir replaces --with-openib-libdir
 * If you specify --with-openib[-libdir] without
   --with-verbs[-libdir], you'll get a "these options have been
   deprecated!" warning, but then they'll act just like
   --with-verbs[--libdir]. 

  '''Sidenote:''' Note that we are not renaming any components at this
  time, nor are we renaming the top-level OMPI_CHECK_OPENIB m4 macro
  (which is pretty strongly tied to the openib BTL and is bastaridzed
  by the ofud BTL).  Note that there will likely be more changes in
  this area coming soon (next week?) when some long-standing changes
  move to the SVN trunk: some openib BTL infrastructure will move to
  ompi/mca/common, and its configury gets split up / refactored.

We extend our philosophy of other --with-<foo> configure options of
--with-verbs to ''all'' verbs-lovin components:

 * If you specify --with-verbs, then all verbs-lovin' components must
   configure successfully (or abort).  This currently means: OOB ud,
   BTL ofud, BTL openib.
 * If you specify --with-verbs=DIR, then all verbs-lovin' component
   must configure successfully (or abort), and will use DIR to find
   verbs headers and libraries.
 * If you specify --without-verbs, then all verbs-lovin' components
   will be ignored.

This commit also fixes a problem where the --with-openib=DIR form
would not use DIR for ''all'' verbs-lovin' components (I think only
BTL openib and BTL ofud used that DIR).  Now all of them do, as does
hwloc (because hwloc has some !OpenFabrics helper functions that
require ibv types from verbs.h).

There's a little new m4 infrastructure worth mentioning:

 * If you create a new verbs-lovin' component (i.e., a component that
   need verbs), your configure.m4 should
   AC_REQUIRE([OPAL_CHECK_VERBS_DIR]). 
 * You can then use three global shell variables: $opal_want_verbs,
   $opal_verbs_dir, $opal_verbs_libdir, which will be set as follows:
   * opal_want_verbs will be "yes" and opal_verbs_dir and
     opal_verbs_libdir will both be set to directory values, '''OR'''
   * opal_want_verbs will be "no" and opal_verbs_dir and
     opal_verbs_libdir will both be set empty

This commit was SVN r26640.
This commit is contained in:
Jeff Squyres 2012-06-22 19:53:56 +00:00
parent 8b693b09af
commit 148ae6d6e3
5 changed files with 210 additions and 38 deletions

20
README
View File

@ -711,21 +711,27 @@ NETWORKING SUPPORT / OPTIONS
look in <mxm directory>/lib and <mxm directory>/lib64, which covers
most cases. This option is only needed for special configurations.
--with-openib=<directory>
Specify the directory where the OpenFabrics (previously known as
OpenIB) libraries and header files are located. This option is
generally only necessary if the OpenFabrics headers and libraries
are not in default compiler/linker search paths.
--with-verbs=<directory>
Specify the directory where the verbs (also know as OpenFabrics, and
previously known as OpenIB) libraries and header files are located.
This option is generally only necessary if the verbs headers and
libraries are not in default compiler/linker search paths.
"OpenFabrics" refers to iWARP-, RoCE- (aka "IBoIP"), and
InfiniBand-based networks.
--with-openib-libdir=<directory>
Look in directory for the OpenFabrics libraries. By default, Open
--with-verbs-libdir=<directory>
Look in directory for the verbs libraries. By default, Open
MPI will look in <openib directory>/lib and <openib
directory>/lib64, which covers most cases. This option is only
needed for special configurations.
--with-openib=<directory>
DEPRECATED synonym for --with-verbs.
--with-openib-libdir=<directory>
DEPRECATED synonym for --with-verbs-libdir.
--with-portals=<directory>
Specify the directory where the Portals libraries and header files
are located. This option is generally only necessary if the Portals

164
config/opal_check_verbs.m4 Normal file
View File

@ -0,0 +1,164 @@
# -*- shell-script -*-
#
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
# University Research and Technology
# Corporation. All rights reserved.
# Copyright (c) 2004-2005 The University of Tennessee and The University
# of Tennessee Research Foundation. All rights
# reserved.
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2006-2012 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2006-2011 Los Alamos National Security, LLC. All rights
# reserved.
# Copyright (c) 2006-2009 Mellanox Technologies. All rights reserved.
# Copyright (c) 2010-2012 Oracle and/or its affiliates. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
#
# $HEADER$
#
# Internal helper macro to look for the verbs libdir
# --------------------------------------------------------
AC_DEFUN([_OPAL_CHECK_VERBS_LIBDIR],[
AS_IF([test -d "$1"],
[AS_IF([test "x`ls $1/libibverbs.* 2> /dev/null`" != "x"],
[opal_verbs_libdir="$1"])
])
])
# Internal helper macro to look for the verbs dir
# --------------------------------------------------------
AC_DEFUN([_OPAL_CHECK_VERBS_DIR],[
AS_IF([test -d "$1"],
[AS_IF([test -f "$1/include/infiniband/verbs.h"],
[opal_verbs_dir="$1"])
])
])
# OPAL_CHECK_VERBS_DIR
# --------------------------------------------------------
# Add --with-verbs options, and if directories are specified,
# sanity check them.
#
# At the end of this macro:
#
# 1. $opal_want_verbs will be set to "yes" and both of
# $opal_verbs_dir and $opal_verbs_libdir will be set
#
# or
#
# 2. $opal_want_verbs will be set to "no" and both of
# $opal_verbs_dir and $opal_verbs_libdir will be empty
#
AC_DEFUN([OPAL_CHECK_VERBS_DIR],[
# Add --with options
AC_ARG_WITH([verbs],
[AC_HELP_STRING([--with-verbs(=DIR)],
[Build verbs support, optionally adding DIR/include, DIR/lib, and DIR/lib64 to the search path for headers and libraries])])
AC_ARG_WITH([verbs-libdir],
[AC_HELP_STRING([--with-verbs-libdir=DIR],
[Search for verbs libraries in DIR])])
# Add deprecated --with options
# JMS REMOVE THESE IN v1.9
AC_ARG_WITH([openib],
[AC_HELP_STRING([--with-openib(=DIR)],
[DEPRECATED synonym for --with-verbs])])
AC_ARG_WITH([openib-libdir],
[AC_HELP_STRING([--with-openib-libdir=DIR],
[DEPRECATED synonym for --with-verbs-libdir])])
# Did we specify both --with-openib* and --with-verbs*?
# JMS REMOVE THESE IN v1.9
AS_IF([test -n "$with_openib" -a -n "$with_verbs"],
[AC_MSG_WARN([Both --with-verbs and --with-openib specified.])
AC_MSG_WARN([--with-openib value ignored.])
with_openib=
])
AS_IF([test -n "$with_openib_libdir" -a -n "$with_verbs_libdir"],
[AC_MSG_WARN([Both --with-verbs-libdir and --with-openib-libdir specified.])
AC_MSG_WARN([--with-openib-libdir value ignored.])
with_openib_libdir=
])
# Did we specify --with-openib* and not --with-verbs*?
# JMS REMOVE THESE IN v1.9
AS_IF([test -n "$with_openib" -a -z "$with_verbs"],
[AC_MSG_WARN([**************************************************])
AC_MSG_WARN([The use of --with-openib is deprecated.])
AC_MSG_WARN([You should use --with-verbs instead.])
AC_MSG_WARN([Proceeding with the assumption you meant])
AC_MSG_WARN([--with-verbs=$with_openib])
AC_MSG_WARN([(sleeping to let you read this message)])
AC_MSG_WARN([**************************************************])
sleep 10
with_verbs=$with_openib
with_openib=
])
AS_IF([test -n "$with_openib_libdir" -a -z "$with_verbs_libdir"],
[AC_MSG_WARN([**************************************************])
AC_MSG_WARN([The use of --with-openib-libdir is deprecated.])
AC_MSG_WARN([You should use --with-verbs-libdir instead.])
AC_MSG_WARN([Proceeding with the assumption you meant])
AC_MSG_WARN([--with-verbs-libdir=$with_openib_libdir])
AC_MSG_WARN([(sleeping to let you read this message)])
AC_MSG_WARN([**************************************************])
sleep 10
with_verbs_libdir=$with_openib_libdir
with_openib_libdir=
])
# Sanity check the --with values
OMPI_CHECK_WITHDIR([verbs], [$with_verbs],
[include/infiniband/verbs.h])
OMPI_CHECK_WITHDIR([verbs-libdir], [$with_verbs_libdir],
[libibverbs.*])
# Set standardized shell variables for OFED lovin' components to
# use. Either both of $opal_verbs_dir and
# $verbs_libdir will be set, or neither will be set.
opal_want_verbs=no
opal_verbs_dir=
AS_IF([test -n "$with_verbs" -a "$with_verbs" != "yes" -a "$with_verbs" != "no"],
[opal_want_verbs=yes
opal_verbs_dir=$with_verbs
])
opal_verbs_libdir=
AS_IF([test -n "$with_verbs_libdir" -a "$with_verbs_libdir" != "yes" -a "$with_verbs_libdir" != "no"],
[opal_want_verbs=yes
opal_verbs_libdir=$with_verbs_libdir
])
# If the top dir was specified but the libdir was not, look for
# it. Note that if the user needs a specific libdir (i.e., if our
# hueristic ordering below is not sufficient), they need to
# specify it.
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib"])])
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib64"])])
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
[_OPAL_CHECK_VERBS_LIBDIR(["$opal_verbs_dir/lib32"])])
AS_IF([test -z "$opal_verbs_libdir" -a -n "$opal_verbs_dir"],
[AC_MSG_WARN([Could not find libibverbs in the usual locations under $opal_verbs_dir])
AC_MSG_ERROR([Cannot continue])
])
# If the libdir was specified, but the top dir was not, look for
# it. Note that if the user needs a specific top dir (i.e., if
# our hueristic below is not sufficient), they need to specify it.
AS_IF([test -z "$opal_verbs" -a -n "$opal_verbs_libdir"],
[_OPAL_CHECK_VERBS_DIR([`dirname "$opal_verbs_libdir"`])])
AS_IF([test -z "$opal_verbs_dir" -a -n "$opal_verbs_libdir"],
[AC_MSG_WARN([Could not find verbs.h in the usual locations under $opal_verbs_dir])
AC_MSG_ERROR([Cannot continue])
])
])

View File

@ -10,7 +10,7 @@
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2006-2010 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2006-2012 Cisco Systems, Inc. All rights reserved.
# Copyright (c) 2006-2011 Los Alamos National Security, LLC. All rights
# reserved.
# Copyright (c) 2006-2009 Mellanox Technologies. All rights reserved.
@ -31,25 +31,17 @@
AC_DEFUN([OMPI_CHECK_OPENIB],[
OPAL_VAR_SCOPE_PUSH([$1_msg])
#
# Openfabrics support
#
AC_ARG_WITH([openib],
[AC_HELP_STRING([--with-openib(=DIR)],
[Build OpenFabrics support, optionally adding DIR/include, DIR/lib, and DIR/lib64 to the search path for headers and libraries])])
OMPI_CHECK_WITHDIR([openib], [$with_openib], [include/infiniband/verbs.h])
AC_ARG_WITH([openib-libdir],
[AC_HELP_STRING([--with-openib-libdir=DIR],
[Search for OpenFabrics libraries in DIR])])
OMPI_CHECK_WITHDIR([openib-libdir], [$with_openib_libdir], [libibverbs.*])
# Setup the --with switches to allow users to specify where
# verbs stuff lives.
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
#
# Add padding to OpenIB header
#
AC_ARG_ENABLE([openib-control-hdr-padding],
[AC_HELP_STRING([--enable-openib-control-hdr-padding],
[Add padding bytes to the openib control header (default:disabled)])])
AC_MSG_CHECKING([if want to add padding to the openib control header])
[Add padding bytes to the openib BTL control header (default:disabled)])])
AC_MSG_CHECKING([if want to add padding to the openib BTL control header])
if test "$enable_openib_control_hdr_padding" = "yes"; then
AC_MSG_RESULT([yes])
ompi_openib_pad_hdr=1
@ -75,14 +67,14 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
esac
fi
AC_DEFINE_UNQUOTED([OMPI_OPENIB_PAD_HDR], [$ompi_openib_pad_hdr],
[Add padding bytes to the openib control header])
[Add padding bytes to the openib BTL control header])
#
# ConnectX XRC support
#
AC_ARG_ENABLE([openib-connectx-xrc],
[AC_HELP_STRING([--enable-openib-connectx-xrc],
[Enable ConnectX XRC support. If you do not have InfiniBand ConnectX adapters, you may disable the ConnectX XRC support. If you do not know which InfiniBand adapter is installed on your cluster, leave this option enabled (default: enabled)])],
[Enable ConnectX XRC support in the openib BTL. If you do not have InfiniBand ConnectX adapters, you may disable the ConnectX XRC support. If you do not know which InfiniBand adapter is installed on your cluster, leave this option enabled (default: enabled)])],
[enable_connectx_xrc="$enableval"], [enable_connectx_xrc="yes"])
#
@ -100,11 +92,7 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
[AC_HELP_STRING([--enable-openib-rdmacm],
[Enable Open Fabrics RDMACM support in openib BTL (default: enabled)])])
AS_IF([test ! -z "$with_openib" -a "$with_openib" != "yes"],
[ompi_check_openib_dir="$with_openib"])
AS_IF([test ! -z "$with_openib_libdir" -a "$with_openib_libdir" != "yes"],
[ompi_check_openib_libdir="$with_openib_libdir"])
AS_IF([test "$with_openib" = "no"],
AS_IF([test "$opal_want_verbs" = "no"],
[ompi_check_openib_happy="no"],
[ompi_check_openib_happy="yes"])
@ -140,8 +128,8 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
[ibverbs],
[ibv_open_device],
[],
[$ompi_check_openib_dir],
[$ompi_check_openib_libdir],
[$opal_verbs_dir],
[$opal_verbs_libdir],
[ompi_check_openib_happy="yes"],
[ompi_check_openib_happy="no"])])
@ -321,9 +309,9 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
AC_MSG_RESULT([no])
fi
AS_IF([test -z "$ompi_check_openib_dir"],
AS_IF([test -z "$opal_verbs_dir"],
[openib_include_dir="/usr/include"],
[openib_include_dir="$ompi_check_openib_dir/include"])
[openib_include_dir="$opal_verbs_dir/include"])
CPPFLAGS="$ompi_check_openib_$1_save_CPPFLAGS -I$openib_include_dir/infiniband"
LDFLAGS="$ompi_check_openib_$1_save_LDFLAGS"
@ -331,12 +319,11 @@ AC_DEFUN([OMPI_CHECK_OPENIB],[
AS_IF([test "$ompi_check_openib_happy" = "yes"],
[$2],
[AS_IF([test ! -z "$with_openib" -a "$with_openib" != "no"],
[AC_MSG_WARN([OpenFabrics support requested (via --with-openib) but not found.])
[AS_IF([test "$opal_want_verbs" = "yes"],
[AC_MSG_WARN([Verbs support requested (via --with-verbs) but not found.])
AC_MSG_WARN([If you are using libibverbs v1.0 (i.e., OFED v1.0 or v1.1), you *MUST* have both the libsysfs headers and libraries installed. Later versions of libibverbs do not require libsysfs.])
AC_MSG_ERROR([Aborting.])])
$3])
OPAL_VAR_SCOPE_POP
])

View File

@ -40,6 +40,9 @@ AC_DEFUN([MCA_opal_hwloc_hwloc142_POST_CONFIG],[
# MCA_hwloc_hwloc142_CONFIG([action-if-found], [action-if-not-found])
# --------------------------------------------------------------------
AC_DEFUN([MCA_opal_hwloc_hwloc142_CONFIG],[
# Hwloc needs to know if we have Verbs support
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
AC_CONFIG_FILES([opal/mca/hwloc/hwloc142/Makefile])
OPAL_VAR_SCOPE_PUSH([HWLOC_VERSION opal_hwloc_hwloc142_save_CPPFLAGS opal_hwloc_hwloc142_save_LDFLAGS opal_hwloc_hwloc142_save_LIBS opal_hwloc_hwloc142_save_cairo opal_hwloc_hwloc142_save_xml opal_hwloc_hwloc142_basedir opal_hwloc_hwloc142_file opal_hwloc_hwloc142_save_cflags])
@ -110,7 +113,11 @@ AC_DEFUN([MCA_opal_hwloc_hwloc142_CONFIG],[
[Version of hwloc])
# Do we have verbs support?
CPPFLAGS_save=$CPPFLAGS
AS_IF([test "$opal_want_verbs" = "yes"],
[CPPFLAGS="-I$opal_verbs_dir/include $CPPFLAGS"])
AC_CHECK_HEADERS([infiniband/verbs.h])
CPPFLAGS=$CPPFLAGS_save
# Set these variables so that the framework m4 knows
# what file to include in opal/mca/hwloc/hwloc.h

View File

@ -10,6 +10,7 @@
# University of Stuttgart. All rights reserved.
# Copyright (c) 2004-2005 The Regents of the University of California.
# All rights reserved.
# Copyright (c) 2012 Cisco Systems, Inc. All rights reserved.
# $COPYRIGHT$
#
# Additional copyrights may follow
@ -20,6 +21,9 @@
# MCA_oob_ud_CONFIG([action-if-found], [action-if-not-found])
# -----------------------------------------------------------
AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
# We need to know if we have verbs support
AC_REQUIRE([OPAL_CHECK_VERBS_DIR])
AC_CONFIG_FILES([orte/mca/oob/ud/Makefile])
# JMS Still have problems with AC_ARG ENABLE not yet having been
@ -34,8 +38,8 @@ AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
[ibverbs],
[ibv_open_device],
[],
[$ompi_check_openib_dir],
[$ompi_check_openib_libdir],
[$opal_verbs_dir],
[$opal_verbs_libdir],
[orte_oob_ud_check_happy=yes],
[orte_oob_ud_check_happy=no])
@ -45,7 +49,11 @@ AC_DEFUN([MCA_orte_oob_ud_CONFIG],[
AS_IF([test "$orte_oob_ud_check_happy" = "yes"],
[$1],
[$2])
[AS_IF([test "$opal_want_verbs" = "yes"],
[AC_MSG_WARN([--with-verbs specified, but cannot build this component])
AC_MSG_ERROR([Cannot continue])
])
$2])
# substitute in the things needed to build this component
AC_SUBST([orte_oob_ud_CFLAGS])