1
1
openmpi/opal/runtime/opal_params.c

121 строка
4.2 KiB
C
Исходник Обычный вид История

/*
* Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
* University Research and Technology
* Corporation. All rights reserved.
* Copyright (c) 2004-2005 The University of Tennessee and The University
* of Tennessee Research Foundation. All rights
* reserved.
* Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
* University of Stuttgart. All rights reserved.
* Copyright (c) 2004-2005 The Regents of the University of California.
* All rights reserved.
* Copyright (c) 2006 Los Alamos National Security, LLC. All rights
* reserved.
Brice Goglin noticed that mpi_paffinity_alone didn't seem to be doing anything for non-MPI apps. Oops! (But before you freak out, gentle reader, note that mpi_paffinity_alone for MPI apps still worked fine) When we made the switchover somewhere in the 1.3 series to have the orted's do processor binding, then stuff like: mpirun --mca mpi_paffinity_alone 1 hostname should have bound hostname to processor 0. But it didn't because of a subtle startup ordering issue: the MCA param registration for opal_paffinity_alone was in the paffinity base (vs. being in opal/runtime/opal_params.c), but it didn't actually get registered until after the global variable opal_paffinity_alone was checked to see if we wanted old-style affinity bindings. Oops. However, for MPI apps, even though the orted didn't do the binding, ompi_mpi_init() would notice that opal_paffinity_alone was set, yet the process didn't seem to be bound. So the MPI process would bind itself (this was done to support the running-without-orteds scenarios). Hence, MPI apps still obeyed mpi_paffinity_alone semantics. But note that the error described above caused the new mpirun switch --report-bindings to not work with mpi_paffinity_alone=1, meaning that the orted would not report the bindings when mpi_paffinity_alone was set to 1 (it ''did'' correctly report bindings if you used --bind-to-core or one of the other binding options). This commit separates out the paffinity base MCA param registration into a small function that can be called at the Right place during the startup sequence. This commit was SVN r22602.
2010-02-11 01:32:00 +03:00
* Copyright (c) 2008-2010 Cisco Systems, Inc. All rights reserved.
- Split the datatype engine into two parts: an MPI specific part in OMPI and a language agnostic part in OPAL. The convertor is completely moved into OPAL. This offers several benefits as described in RFC http://www.open-mpi.org/community/lists/devel/2009/07/6387.php namely: - Fewer basic types (int* and float* types, boolean and wchar - Fixing naming scheme to ompi-nomenclature. - Usability outside of the ompi-layer. - Due to the fixed nature of simple opal types, their information is completely known at compile time and therefore constified - With fewer datatypes (22), the actual sizes of bit-field types may be reduced from 64 to 32 bits, allowing reorganizing the opal_datatype structure, eliminating holes and keeping data required in convertor (upon send/recv) in one cacheline... This has implications to the convertor-datastructure and other parts of the code. - Several performance tests have been run, the netpipe latency does not change with this patch on Linux/x86-64 on the smoky cluster. - Extensive tests have been done to verify correctness (no new regressions) using: 1. mpi_test_suite on linux/x86-64 using clean ompi-trunk and ompi-ddt: a. running both trunk and ompi-ddt resulted in no differences (except for MPI_SHORT_INT and MPI_TYPE_MIX_LB_UB do now run correctly). b. with --enable-memchecker and running under valgrind (one buglet when run with static found in test-suite, commited) 2. ibm testsuite on linux/x86-64 using clean ompi-trunk and ompi-ddt: all passed (except for the dynamic/ tests failed!! as trunk/MTT) 3. compilation and usage of HDF5 tests on Jaguar using PGI and PathScale compilers. 4. compilation and usage on Scicortex. - Please note, that for the heterogeneous case, (-m32 compiled binaries/ompi), neither ompi-trunk, nor ompi-ddt branch would successfully launch. This commit was SVN r21641.
2009-07-13 08:56:31 +04:00
* Copyright (c) 2009 Oak Ridge National Labs. All rights reserved.
* Copyright (c) 2010 Los Alamos National Security, LLC.
* All rights reserved.
* $COPYRIGHT$
*
* Additional copyrights may follow
*
* $HEADER$
*/
#include "opal_config.h"
#include <time.h>
#ifdef HAVE_SIGNAL_H
#include <signal.h>
#endif
#include "opal/constants.h"
#include "opal/runtime/opal.h"
- Split the datatype engine into two parts: an MPI specific part in OMPI and a language agnostic part in OPAL. The convertor is completely moved into OPAL. This offers several benefits as described in RFC http://www.open-mpi.org/community/lists/devel/2009/07/6387.php namely: - Fewer basic types (int* and float* types, boolean and wchar - Fixing naming scheme to ompi-nomenclature. - Usability outside of the ompi-layer. - Due to the fixed nature of simple opal types, their information is completely known at compile time and therefore constified - With fewer datatypes (22), the actual sizes of bit-field types may be reduced from 64 to 32 bits, allowing reorganizing the opal_datatype structure, eliminating holes and keeping data required in convertor (upon send/recv) in one cacheline... This has implications to the convertor-datastructure and other parts of the code. - Several performance tests have been run, the netpipe latency does not change with this patch on Linux/x86-64 on the smoky cluster. - Extensive tests have been done to verify correctness (no new regressions) using: 1. mpi_test_suite on linux/x86-64 using clean ompi-trunk and ompi-ddt: a. running both trunk and ompi-ddt resulted in no differences (except for MPI_SHORT_INT and MPI_TYPE_MIX_LB_UB do now run correctly). b. with --enable-memchecker and running under valgrind (one buglet when run with static found in test-suite, commited) 2. ibm testsuite on linux/x86-64 using clean ompi-trunk and ompi-ddt: all passed (except for the dynamic/ tests failed!! as trunk/MTT) 3. compilation and usage of HDF5 tests on Jaguar using PGI and PathScale compilers. 4. compilation and usage on Scicortex. - Please note, that for the heterogeneous case, (-m32 compiled binaries/ompi), neither ompi-trunk, nor ompi-ddt branch would successfully launch. This commit was SVN r21641.
2009-07-13 08:56:31 +04:00
#include "opal/datatype/opal_datatype.h"
#include "opal/mca/base/mca_base_param.h"
#include "opal/threads/mutex.h"
#include "opal/threads/threads.h"
Brice Goglin noticed that mpi_paffinity_alone didn't seem to be doing anything for non-MPI apps. Oops! (But before you freak out, gentle reader, note that mpi_paffinity_alone for MPI apps still worked fine) When we made the switchover somewhere in the 1.3 series to have the orted's do processor binding, then stuff like: mpirun --mca mpi_paffinity_alone 1 hostname should have bound hostname to processor 0. But it didn't because of a subtle startup ordering issue: the MCA param registration for opal_paffinity_alone was in the paffinity base (vs. being in opal/runtime/opal_params.c), but it didn't actually get registered until after the global variable opal_paffinity_alone was checked to see if we wanted old-style affinity bindings. Oops. However, for MPI apps, even though the orted didn't do the binding, ompi_mpi_init() would notice that opal_paffinity_alone was set, yet the process didn't seem to be bound. So the MPI process would bind itself (this was done to support the running-without-orteds scenarios). Hence, MPI apps still obeyed mpi_paffinity_alone semantics. But note that the error described above caused the new mpirun switch --report-bindings to not work with mpi_paffinity_alone=1, meaning that the orted would not report the bindings when mpi_paffinity_alone was set to 1 (it ''did'' correctly report bindings if you used --bind-to-core or one of the other binding options). This commit separates out the paffinity base MCA param registration into a small function that can be called at the Right place during the startup sequence. This commit was SVN r22602.
2010-02-11 01:32:00 +03:00
#include "opal/mca/paffinity/base/base.h"
#include "opal/mca/shmem/base/base.h"
int opal_register_params(void)
{
Brice Goglin noticed that mpi_paffinity_alone didn't seem to be doing anything for non-MPI apps. Oops! (But before you freak out, gentle reader, note that mpi_paffinity_alone for MPI apps still worked fine) When we made the switchover somewhere in the 1.3 series to have the orted's do processor binding, then stuff like: mpirun --mca mpi_paffinity_alone 1 hostname should have bound hostname to processor 0. But it didn't because of a subtle startup ordering issue: the MCA param registration for opal_paffinity_alone was in the paffinity base (vs. being in opal/runtime/opal_params.c), but it didn't actually get registered until after the global variable opal_paffinity_alone was checked to see if we wanted old-style affinity bindings. Oops. However, for MPI apps, even though the orted didn't do the binding, ompi_mpi_init() would notice that opal_paffinity_alone was set, yet the process didn't seem to be bound. So the MPI process would bind itself (this was done to support the running-without-orteds scenarios). Hence, MPI apps still obeyed mpi_paffinity_alone semantics. But note that the error described above caused the new mpirun switch --report-bindings to not work with mpi_paffinity_alone=1, meaning that the orted would not report the bindings when mpi_paffinity_alone was set to 1 (it ''did'' correctly report bindings if you used --bind-to-core or one of the other binding options). This commit separates out the paffinity base MCA param registration into a small function that can be called at the Right place during the startup sequence. This commit was SVN r22602.
2010-02-11 01:32:00 +03:00
int ret;
/*
* This string is going to be used in opal/util/stacktrace.c
*/
{
char *string = NULL;
int j;
int signals[] = {
#ifdef SIGABRT
SIGABRT,
#endif
#ifdef SIGBUS
SIGBUS,
#endif
#ifdef SIGFPE
SIGFPE,
#endif
#ifdef SIGSEGV
SIGSEGV,
#endif
-1
};
for (j = 0 ; signals[j] != -1 ; ++j) {
if (j == 0) {
asprintf(&string, "%d", signals[j]);
} else {
char *tmp;
asprintf(&tmp, "%s,%d", string, signals[j]);
free(string);
string = tmp;
}
}
mca_base_param_reg_string_name("opal", "signal",
"Comma-delimited list of integer signal numbers to Open MPI to attempt to intercept. Upon receipt of the intercepted signal, Open MPI will display a stack trace and abort. Open MPI will *not* replace signals if handlers are already installed by the time MPI_INIT is invoked. Optionally append \":complain\" to any signal number in the comma-delimited list to make Open MPI complain if it detects another signal handler (and therefore does not insert its own).",
false, false, string, NULL);
free(string);
}
#if OPAL_ENABLE_DEBUG
mca_base_param_reg_int_name("opal", "progress_debug",
"Set to non-zero to debug progress engine features",
false, false, 0, NULL);
{
int value;
mca_base_param_reg_int_name("opal", "debug_locks",
"Debug mutex usage within Open MPI. On a "
"non-threaded build, this enables integer counters and "
"warning messages when double-locks are detected.",
false, false, 0, &value);
if (value) opal_mutex_check_locks = true;
mca_base_param_reg_int_name("opal", "debug_threads",
"Debug thread usage within OPAL. Reports out "
"when threads are acquired and released.",
false, false, 0, &value);
if (value) opal_debug_threads = true;
}
#endif
- Split the datatype engine into two parts: an MPI specific part in OMPI and a language agnostic part in OPAL. The convertor is completely moved into OPAL. This offers several benefits as described in RFC http://www.open-mpi.org/community/lists/devel/2009/07/6387.php namely: - Fewer basic types (int* and float* types, boolean and wchar - Fixing naming scheme to ompi-nomenclature. - Usability outside of the ompi-layer. - Due to the fixed nature of simple opal types, their information is completely known at compile time and therefore constified - With fewer datatypes (22), the actual sizes of bit-field types may be reduced from 64 to 32 bits, allowing reorganizing the opal_datatype structure, eliminating holes and keeping data required in convertor (upon send/recv) in one cacheline... This has implications to the convertor-datastructure and other parts of the code. - Several performance tests have been run, the netpipe latency does not change with this patch on Linux/x86-64 on the smoky cluster. - Extensive tests have been done to verify correctness (no new regressions) using: 1. mpi_test_suite on linux/x86-64 using clean ompi-trunk and ompi-ddt: a. running both trunk and ompi-ddt resulted in no differences (except for MPI_SHORT_INT and MPI_TYPE_MIX_LB_UB do now run correctly). b. with --enable-memchecker and running under valgrind (one buglet when run with static found in test-suite, commited) 2. ibm testsuite on linux/x86-64 using clean ompi-trunk and ompi-ddt: all passed (except for the dynamic/ tests failed!! as trunk/MTT) 3. compilation and usage of HDF5 tests on Jaguar using PGI and PathScale compilers. 4. compilation and usage on Scicortex. - Please note, that for the heterogeneous case, (-m32 compiled binaries/ompi), neither ompi-trunk, nor ompi-ddt branch would successfully launch. This commit was SVN r21641.
2009-07-13 08:56:31 +04:00
/* The ddt engine has a few parameters */
Brice Goglin noticed that mpi_paffinity_alone didn't seem to be doing anything for non-MPI apps. Oops! (But before you freak out, gentle reader, note that mpi_paffinity_alone for MPI apps still worked fine) When we made the switchover somewhere in the 1.3 series to have the orted's do processor binding, then stuff like: mpirun --mca mpi_paffinity_alone 1 hostname should have bound hostname to processor 0. But it didn't because of a subtle startup ordering issue: the MCA param registration for opal_paffinity_alone was in the paffinity base (vs. being in opal/runtime/opal_params.c), but it didn't actually get registered until after the global variable opal_paffinity_alone was checked to see if we wanted old-style affinity bindings. Oops. However, for MPI apps, even though the orted didn't do the binding, ompi_mpi_init() would notice that opal_paffinity_alone was set, yet the process didn't seem to be bound. So the MPI process would bind itself (this was done to support the running-without-orteds scenarios). Hence, MPI apps still obeyed mpi_paffinity_alone semantics. But note that the error described above caused the new mpirun switch --report-bindings to not work with mpi_paffinity_alone=1, meaning that the orted would not report the bindings when mpi_paffinity_alone was set to 1 (it ''did'' correctly report bindings if you used --bind-to-core or one of the other binding options). This commit separates out the paffinity base MCA param registration into a small function that can be called at the Right place during the startup sequence. This commit was SVN r22602.
2010-02-11 01:32:00 +03:00
ret = opal_datatype_register_params();
if (OPAL_SUCCESS != ret) {
return ret;
}
/* shmem base also has a few parameters */
ret = opal_shmem_base_register_params();
if (OPAL_SUCCESS != ret) {
return ret;
}
Brice Goglin noticed that mpi_paffinity_alone didn't seem to be doing anything for non-MPI apps. Oops! (But before you freak out, gentle reader, note that mpi_paffinity_alone for MPI apps still worked fine) When we made the switchover somewhere in the 1.3 series to have the orted's do processor binding, then stuff like: mpirun --mca mpi_paffinity_alone 1 hostname should have bound hostname to processor 0. But it didn't because of a subtle startup ordering issue: the MCA param registration for opal_paffinity_alone was in the paffinity base (vs. being in opal/runtime/opal_params.c), but it didn't actually get registered until after the global variable opal_paffinity_alone was checked to see if we wanted old-style affinity bindings. Oops. However, for MPI apps, even though the orted didn't do the binding, ompi_mpi_init() would notice that opal_paffinity_alone was set, yet the process didn't seem to be bound. So the MPI process would bind itself (this was done to support the running-without-orteds scenarios). Hence, MPI apps still obeyed mpi_paffinity_alone semantics. But note that the error described above caused the new mpirun switch --report-bindings to not work with mpi_paffinity_alone=1, meaning that the orted would not report the bindings when mpi_paffinity_alone was set to 1 (it ''did'' correctly report bindings if you used --bind-to-core or one of the other binding options). This commit separates out the paffinity base MCA param registration into a small function that can be called at the Right place during the startup sequence. This commit was SVN r22602.
2010-02-11 01:32:00 +03:00
/* Paffinity base also has some parameters */
return opal_paffinity_base_register_params();
}