2006-07-10 21:25:33 +00:00
|
|
|
# -*- text -*-
|
|
|
|
|
#
|
|
|
|
|
# Copyright (c) 2004-2005 The Trustees of Indiana University and Indiana
|
|
|
|
|
# University Research and Technology
|
|
|
|
|
# Corporation. All rights reserved.
|
|
|
|
|
# Copyright (c) 2004-2005 The University of Tennessee and The University
|
|
|
|
|
# of Tennessee Research Foundation. All rights
|
|
|
|
|
# reserved.
|
2015-06-23 20:59:57 -07:00
|
|
|
# Copyright (c) 2004-2005 High Performance Computing Center Stuttgart,
|
2006-07-10 21:25:33 +00:00
|
|
|
# University of Stuttgart. All rights reserved.
|
|
|
|
|
# Copyright (c) 2004-2005 The Regents of the University of California.
|
|
|
|
|
# All rights reserved.
|
2015-12-14 13:02:41 -05:00
|
|
|
# Copyright (c) 2011-2015 Cisco Systems, Inc. All rights reserved.
|
2011-12-02 14:10:08 +00:00
|
|
|
# Copyright (c) 2011 Los Alamos National Security, LLC.
|
2015-06-23 20:59:57 -07:00
|
|
|
# All rights reserved.
|
2014-01-15 14:48:39 +00:00
|
|
|
# Copyright (c) 2014 Intel, Inc. All rights reserved.
|
2006-07-10 21:25:33 +00:00
|
|
|
# $COPYRIGHT$
|
2015-06-23 20:59:57 -07:00
|
|
|
#
|
2006-07-10 21:25:33 +00:00
|
|
|
# Additional copyrights may follow
|
2015-06-23 20:59:57 -07:00
|
|
|
#
|
2006-07-10 21:25:33 +00:00
|
|
|
# $HEADER$
|
|
|
|
|
#
|
|
|
|
|
# This is the US/English general help file for Open RTE's orterun.
|
|
|
|
|
#
|
|
|
|
|
[orte-rmaps-base:alloc-error]
|
2015-06-23 20:59:57 -07:00
|
|
|
There are not enough slots available in the system to satisfy the %d slots
|
2006-07-10 21:25:33 +00:00
|
|
|
that were requested by the application:
|
|
|
|
|
%s
|
|
|
|
|
|
|
|
|
|
Either request fewer slots for your application, or make more slots available
|
|
|
|
|
for use.
|
|
|
|
|
[orte-rmaps-base:not-all-mapped-alloc]
|
|
|
|
|
Some of the requested hosts are not included in the current allocation for the
|
|
|
|
|
application:
|
|
|
|
|
%s
|
|
|
|
|
The requested hosts were:
|
|
|
|
|
%s
|
|
|
|
|
|
2015-06-23 20:59:57 -07:00
|
|
|
Verify that you have mapped the allocated resources properly using the
|
2008-02-28 01:57:57 +00:00
|
|
|
--host or --hostfile specification.
|
2006-07-10 21:25:33 +00:00
|
|
|
[orte-rmaps-base:no-mapped-node]
|
2015-06-23 20:59:57 -07:00
|
|
|
There are no allocated resources for the application:
|
2006-07-10 21:25:33 +00:00
|
|
|
%s
|
|
|
|
|
that match the requested mapping:
|
2013-03-28 16:52:10 +00:00
|
|
|
%s: %s
|
2006-07-10 21:25:33 +00:00
|
|
|
|
2015-06-23 20:59:57 -07:00
|
|
|
Verify that you have mapped the allocated resources properly for the
|
2013-03-28 16:52:10 +00:00
|
|
|
indicated specification.
|
2006-10-26 21:46:18 +00:00
|
|
|
[orte-rmaps-base:nolocal-no-available-resources]
|
|
|
|
|
There are no available nodes allocated to this job. This could be because
|
|
|
|
|
no nodes were found or all the available nodes were already used.
|
|
|
|
|
|
2015-06-23 20:59:57 -07:00
|
|
|
Note that since the -nolocal option was given no processes can be
|
2006-10-26 21:46:18 +00:00
|
|
|
launched on the local node.
|
|
|
|
|
[orte-rmaps-base:no-available-resources]
|
2010-07-01 19:39:31 +00:00
|
|
|
No nodes are available for this job, either due to a failure to
|
|
|
|
|
allocate nodes to the job, or allocated nodes being marked
|
|
|
|
|
as unavailable (e.g., down, rebooting, or a process attempting
|
|
|
|
|
to be relocated to another node when none are available).
|
2006-10-26 21:46:18 +00:00
|
|
|
[orte-rmaps-base:all-available-resources-used]
|
|
|
|
|
All nodes which are allocated for this job are already filled.
|
2008-02-28 01:57:57 +00:00
|
|
|
#
|
|
|
|
|
[out-of-vpids]
|
|
|
|
|
The system has exhausted its available ranks - the application is attempting
|
|
|
|
|
to spawn too many daemons and will be aborted.
|
|
|
|
|
|
|
|
|
|
This may be resolved by increasing the number of available ranks by
|
|
|
|
|
re-configuring with the --enable-jumbo-apps option, and then
|
|
|
|
|
re-building the application.
|
2009-09-07 03:36:10 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:too-many-procs]
|
|
|
|
|
Your job has requested a conflicting number of processes for the
|
|
|
|
|
application:
|
|
|
|
|
|
|
|
|
|
App: %s
|
|
|
|
|
number of procs: %d
|
|
|
|
|
|
|
|
|
|
This is more processes than we can launch under the following
|
|
|
|
|
additional directives and conditions:
|
|
|
|
|
|
|
|
|
|
%s: %d
|
|
|
|
|
%s: %d
|
2006-10-26 21:46:18 +00:00
|
|
|
|
2009-09-07 03:36:10 +00:00
|
|
|
Please revise the conflict and try again.
|
2009-10-13 04:19:07 +00:00
|
|
|
#
|
|
|
|
|
[too-many-cpus-per-rank]
|
|
|
|
|
Your job has requested more cpus per process(rank) than there
|
|
|
|
|
are cpus in a socket:
|
|
|
|
|
|
|
|
|
|
Cpus/rank: %d
|
|
|
|
|
#cpus/socket: %d
|
|
|
|
|
|
|
|
|
|
Please correct one or both of these values and try again.
|
2011-02-15 23:24:31 +00:00
|
|
|
#
|
|
|
|
|
[failed-map]
|
|
|
|
|
Your job failed to map. Either no mapper was available, or none
|
|
|
|
|
of the available mappers was able to perform the requested
|
|
|
|
|
mapping operation. This can happen if you request a map type
|
|
|
|
|
(e.g., loadbalance) and the corresponding mapper was not built.
|
2014-12-08 08:03:53 -08:00
|
|
|
|
|
|
|
|
Mapper result: %s
|
|
|
|
|
#procs mapped: %d
|
|
|
|
|
#nodes assigned: %d
|
|
|
|
|
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#
|
|
|
|
|
[unrecognized-policy]
|
|
|
|
|
The specified %s policy is not recognized:
|
|
|
|
|
|
|
|
|
|
Policy: %s
|
|
|
|
|
|
|
|
|
|
Please check for a typo or ensure that the option is a supported
|
|
|
|
|
one.
|
|
|
|
|
#
|
|
|
|
|
[redefining-policy]
|
|
|
|
|
Conflicting directives for %s policy are causing the policy
|
|
|
|
|
to be redefined:
|
|
|
|
|
|
|
|
|
|
New policy: %s
|
|
|
|
|
Prior policy: %s
|
|
|
|
|
|
|
|
|
|
Please check that only one policy is defined.
|
|
|
|
|
#
|
|
|
|
|
[rmaps:binding-target-not-found]
|
|
|
|
|
A request was made to bind to %s, but an appropriate target could not
|
|
|
|
|
be found on node %s.
|
|
|
|
|
#
|
|
|
|
|
[rmaps:binding-overload]
|
|
|
|
|
A request was made to bind to that would result in binding more
|
|
|
|
|
processes than cpus on a resource:
|
|
|
|
|
|
2014-06-14 15:38:32 +00:00
|
|
|
Bind to: %s
|
|
|
|
|
Node: %s
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#processes: %d
|
2014-06-14 15:38:32 +00:00
|
|
|
#cpus: %d
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
|
|
|
|
|
You can override this protection by adding the "overload-allowed"
|
|
|
|
|
option to your binding directive.
|
|
|
|
|
#
|
|
|
|
|
[rmaps:no-topology]
|
2012-01-27 12:21:45 +00:00
|
|
|
A mapping directive was given that requires knowledge of
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
a remote node's topology. However, no topology info is
|
|
|
|
|
available for the following node:
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
|
|
|
|
|
The job cannot be executed under this condition. Please either
|
2012-01-27 12:21:45 +00:00
|
|
|
remove the directive or investigate the lack of topology info.
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:no-available-cpus]
|
|
|
|
|
While computing bindings, we found no available cpus on
|
|
|
|
|
the following node:
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
|
|
|
|
|
Please check your allocation.
|
|
|
|
|
#
|
|
|
|
|
[rmaps:cpubind-not-supported]
|
|
|
|
|
A request was made to bind a process, but at least one node does NOT
|
|
|
|
|
support binding processes to cpus.
|
|
|
|
|
|
2015-12-14 13:02:41 -05:00
|
|
|
Node: %s
|
|
|
|
|
|
|
|
|
|
Open MPI uses the "hwloc" library to perform process and memory
|
|
|
|
|
binding. This error message means that hwloc has indicated that
|
|
|
|
|
processor binding support is not available on this machine.
|
|
|
|
|
|
|
|
|
|
On OS X, processor and memory binding is not available at all (i.e.,
|
|
|
|
|
the OS does not expose this functionality).
|
|
|
|
|
|
|
|
|
|
On Linux, lack of the functionality can mean that you are on a
|
|
|
|
|
platform where processor and memory affinity is not supported in Linux
|
|
|
|
|
itself, or that hwloc was built without NUMA and/or processor affinity
|
|
|
|
|
support. When building hwloc (which, depending on your Open MPI
|
|
|
|
|
installation, may be embedded in Open MPI itself), it is important to
|
|
|
|
|
have the libnuma header and library files available. Different linux
|
|
|
|
|
distributions package these files under different names; look for
|
|
|
|
|
packages with the word "numa" in them. You may also need a developer
|
|
|
|
|
version of the package (e.g., with "dev" or "devel" in the name) to
|
|
|
|
|
obtain the relevant header files.
|
|
|
|
|
|
|
|
|
|
If you are getting this message on a non-OS X, non-Linux platform,
|
|
|
|
|
then hwloc does not support processor / memory affinity on this
|
|
|
|
|
platform. If the OS/platform does actually support processor / memory
|
|
|
|
|
affinity, then you should contact the hwloc maintainers:
|
|
|
|
|
https://github.com/open-mpi/hwloc.
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:membind-not-supported]
|
|
|
|
|
WARNING: a request was made to bind a process. While the system
|
|
|
|
|
supports binding the process itself, at least one node does NOT
|
|
|
|
|
support binding memory to the process location.
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
|
2015-12-14 13:02:41 -05:00
|
|
|
Open MPI uses the "hwloc" library to perform process and memory
|
|
|
|
|
binding. This error message means that hwloc has indicated that
|
|
|
|
|
processor binding support is not available on this machine.
|
|
|
|
|
|
|
|
|
|
On OS X, processor and memory binding is not available at all (i.e.,
|
|
|
|
|
the OS does not expose this functionality).
|
|
|
|
|
|
|
|
|
|
On Linux, lack of the functionality can mean that you are on a
|
|
|
|
|
platform where processor and memory affinity is not supported in Linux
|
|
|
|
|
itself, or that hwloc was built without NUMA and/or processor affinity
|
|
|
|
|
support. When building hwloc (which, depending on your Open MPI
|
|
|
|
|
installation, may be embedded in Open MPI itself), it is important to
|
|
|
|
|
have the libnuma header and library files available. Different linux
|
|
|
|
|
distributions package these files under different names; look for
|
|
|
|
|
packages with the word "numa" in them. You may also need a developer
|
|
|
|
|
version of the package (e.g., with "dev" or "devel" in the name) to
|
|
|
|
|
obtain the relevant header files.
|
|
|
|
|
|
|
|
|
|
If you are getting this message on a non-OS X, non-Linux platform,
|
|
|
|
|
then hwloc does not support processor / memory affinity on this
|
|
|
|
|
platform. If the OS/platform does actually support processor / memory
|
|
|
|
|
affinity, then you should contact the hwloc maintainers:
|
|
|
|
|
https://github.com/open-mpi/hwloc.
|
|
|
|
|
|
|
|
|
|
This is a warning only; your job will continue, though performance may
|
|
|
|
|
be degraded.
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:membind-not-supported-fatal]
|
|
|
|
|
A request was made to bind a process. While the system
|
|
|
|
|
supports binding the process itself, at least one node does NOT
|
|
|
|
|
support binding memory to the process location.
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
|
2015-12-14 13:02:41 -05:00
|
|
|
Open MPI uses the "hwloc" library to perform process and memory
|
|
|
|
|
binding. This error message means that hwloc has indicated that
|
|
|
|
|
processor binding support is not available on this machine.
|
|
|
|
|
|
|
|
|
|
On OS X, processor and memory binding is not available at all (i.e.,
|
|
|
|
|
the OS does not expose this functionality).
|
|
|
|
|
|
|
|
|
|
On Linux, lack of the functionality can mean that you are on a
|
|
|
|
|
platform where processor and memory affinity is not supported in Linux
|
|
|
|
|
itself, or that hwloc was built without NUMA and/or processor affinity
|
|
|
|
|
support. When building hwloc (which, depending on your Open MPI
|
|
|
|
|
installation, may be embedded in Open MPI itself), it is important to
|
|
|
|
|
have the libnuma header and library files available. Different linux
|
|
|
|
|
distributions package these files under different names; look for
|
|
|
|
|
packages with the word "numa" in them. You may also need a developer
|
|
|
|
|
version of the package (e.g., with "dev" or "devel" in the name) to
|
|
|
|
|
obtain the relevant header files.
|
|
|
|
|
|
|
|
|
|
If you are getting this message on a non-OS X, non-Linux platform,
|
|
|
|
|
then hwloc does not support processor / memory affinity on this
|
|
|
|
|
platform. If the OS/platform does actually support processor / memory
|
|
|
|
|
affinity, then you should contact the hwloc maintainers:
|
|
|
|
|
https://github.com/open-mpi/hwloc.
|
|
|
|
|
|
|
|
|
|
The provided memory binding policy requires that Open MPI abort the
|
|
|
|
|
job at this time.
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:no-bindable-objects]
|
|
|
|
|
No bindable objects of the specified type were available
|
|
|
|
|
on at least one node:
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
Target: %s
|
|
|
|
|
#
|
|
|
|
|
[rmaps:unknown-binding-level]
|
|
|
|
|
Unknown binding level:
|
2011-02-15 23:24:31 +00:00
|
|
|
|
At long last, the fabled revision to the affinity system has arrived. A more detailed explanation of how this all works will be presented here:
https://svn.open-mpi.org/trac/ompi/wiki/ProcessPlacement
The wiki page is incomplete at the moment, but I hope to complete it over the next few days. I will provide updates on the devel list. As the wiki page states, the default and most commonly used options remain unchanged (except as noted below). New, esoteric and complex options have been added, but unless you are a true masochist, you are unlikely to use many of them beyond perhaps an initial curiosity-motivated experimentation.
In a nutshell, this commit revamps the map/rank/bind procedure to take into account topology info on the compute nodes. I have, for the most part, preserved the default behaviors, with three notable exceptions:
1. I have at long last bowed my head in submission to the system admin's of managed clusters. For years, they have complained about our default of allowing users to oversubscribe nodes - i.e., to run more processes on a node than allocated slots. Accordingly, I have modified the default behavior: if you are running off of hostfile/dash-host allocated nodes, then the default is to allow oversubscription. If you are running off of RM-allocated nodes, then the default is to NOT allow oversubscription. Flags to override these behaviors are provided, so this only affects the default behavior.
2. both cpus/rank and stride have been removed. The latter was demanded by those who didn't understand the purpose behind it - and I agreed as the users who requested it are no longer using it. The former was removed temporarily pending implementation.
3. vm launch is now the sole method for starting OMPI. It was just too darned hard to maintain multiple launch procedures - maybe someday, provided someone can demonstrate a reason to do so.
As Jeff stated, it is impossible to fully test a change of this size. I have tested it on Linux and Mac, covering all the default and simple options, singletons, and comm_spawn. That said, I'm sure others will find problems, so I'll be watching MTT results until this stabilizes.
This commit was SVN r25476.
2011-11-15 03:40:11 +00:00
|
|
|
Target: %s
|
|
|
|
|
Cache level: %u
|
2011-12-02 14:10:08 +00:00
|
|
|
#
|
|
|
|
|
[orte-rmaps-base:missing-daemon]
|
|
|
|
|
While attempting to build a map of this job, a node
|
|
|
|
|
was detected to be missing a daemon:
|
|
|
|
|
|
|
|
|
|
Node: %s
|
|
|
|
|
|
|
|
|
|
This usually indicates a mismatch between what the
|
|
|
|
|
allocation provided for the node name versus what was
|
|
|
|
|
actually found on the node.
|
2012-08-04 21:52:36 +00:00
|
|
|
#
|
|
|
|
|
[orte-rmaps-base:no-objects]
|
|
|
|
|
No objects of the specified type were found on at least one node:
|
|
|
|
|
|
|
|
|
|
Type: %s
|
|
|
|
|
Node: %s
|
|
|
|
|
|
|
|
|
|
The map cannot be done as specified.
|
2013-06-27 03:04:50 +00:00
|
|
|
#
|
|
|
|
|
[topo-file]
|
|
|
|
|
A topology file was given for the compute nodes, but
|
|
|
|
|
we were unable to correctly process it. Common errors
|
|
|
|
|
include incorrectly specifying the path to the file,
|
|
|
|
|
or the file being generated in a way that is incompatible
|
|
|
|
|
with the version of hwloc being used by OMPI.
|
|
|
|
|
|
|
|
|
|
File: %s
|
|
|
|
|
|
|
|
|
|
Please correct the problem and try again.
|
2014-01-15 14:48:39 +00:00
|
|
|
#
|
|
|
|
|
[deprecated]
|
After a lot of pain, I've managed to resolve the problem of conflicting mapping directives caused by mismatched MCA params - i.e., where someone has one variant of an MCA param (e.g., rmaps_base_mapping_policy) in their default MCA param file, and then specifies another variant (e.g., --npernode) on the command line. I can't fully resolve the problem as there is no way to know precisely what the user meant - we can only guess which param was really intended since the MCA param system
can't apply its normal precedence rules.
So...print a big "deprecated" warning for the old params and error out if a conflict is detected. I know that isn't what people really wanted, but it's the best we
can do. If only the old style param is given, then process it after the warning.
Extend the current map-by param to add support for ppr and cpus-per-proc, adding the latter to the list of allowed modifiers using "pe=n" for processing elements/proc. Thus, you can map-by socket:pe=2,oversubscribe to map by socket, binding 2 processing elements/process, with oversubscription allowed. Or you can map-by ppr:2:socket:pe=4 to map two processes to every socket in the allocation, binding each process to 4 processing elements.
For those wondering, a processing element is defined as a hwthread if --use-hwthreads-as-cpus is given, or else as a core.
Refs trac:4117
This commit was SVN r30620.
The following Trac tickets were found above:
Ticket 4117 --> https://svn.open-mpi.org/trac/ompi/ticket/4117
2014-02-07 21:25:40 +00:00
|
|
|
The following command line options and corresponding MCA parameter have
|
2014-01-22 12:17:14 +00:00
|
|
|
been deprecated and replaced as follows:
|
2014-01-15 14:48:39 +00:00
|
|
|
|
After a lot of pain, I've managed to resolve the problem of conflicting mapping directives caused by mismatched MCA params - i.e., where someone has one variant of an MCA param (e.g., rmaps_base_mapping_policy) in their default MCA param file, and then specifies another variant (e.g., --npernode) on the command line. I can't fully resolve the problem as there is no way to know precisely what the user meant - we can only guess which param was really intended since the MCA param system
can't apply its normal precedence rules.
So...print a big "deprecated" warning for the old params and error out if a conflict is detected. I know that isn't what people really wanted, but it's the best we
can do. If only the old style param is given, then process it after the warning.
Extend the current map-by param to add support for ppr and cpus-per-proc, adding the latter to the list of allowed modifiers using "pe=n" for processing elements/proc. Thus, you can map-by socket:pe=2,oversubscribe to map by socket, binding 2 processing elements/process, with oversubscription allowed. Or you can map-by ppr:2:socket:pe=4 to map two processes to every socket in the allocation, binding each process to 4 processing elements.
For those wondering, a processing element is defined as a hwthread if --use-hwthreads-as-cpus is given, or else as a core.
Refs trac:4117
This commit was SVN r30620.
The following Trac tickets were found above:
Ticket 4117 --> https://svn.open-mpi.org/trac/ompi/ticket/4117
2014-02-07 21:25:40 +00:00
|
|
|
Command line options:
|
2014-01-15 14:48:39 +00:00
|
|
|
Deprecated: %s
|
|
|
|
|
Replacement: %s
|
|
|
|
|
|
2014-01-22 12:17:14 +00:00
|
|
|
Equivalent MCA parameter:
|
|
|
|
|
Deprecated: %s
|
2014-01-15 14:48:39 +00:00
|
|
|
Replacement: %s
|
|
|
|
|
|
2014-01-22 12:17:14 +00:00
|
|
|
The deprecated forms *will* disappear in a future version of Open MPI.
|
2014-01-15 14:48:39 +00:00
|
|
|
Please update to the new syntax.
|
2014-01-27 22:40:51 +00:00
|
|
|
#
|
|
|
|
|
[mismatch-binding]
|
|
|
|
|
A request for multiple cpus-per-proc was given, but a conflicting binding
|
|
|
|
|
policy was specified:
|
|
|
|
|
|
|
|
|
|
#cpus-per-proc: %d
|
|
|
|
|
type of cpus: %s
|
|
|
|
|
binding policy given: %s
|
|
|
|
|
|
|
|
|
|
The correct binding policy for the given type of cpu is:
|
|
|
|
|
|
|
|
|
|
correct binding policy: %s
|
|
|
|
|
|
|
|
|
|
This is the binding policy we would apply by default for this
|
|
|
|
|
situation, so no binding need be specified. Please correct the
|
|
|
|
|
situation and try again.
|
|
|
|
|
#
|
|
|
|
|
[mapping-too-low]
|
|
|
|
|
A request for multiple cpus-per-proc was given, but a directive
|
2014-03-24 15:47:29 +00:00
|
|
|
was also give to map to an object level that has less cpus than
|
|
|
|
|
requested ones:
|
2014-01-27 22:40:51 +00:00
|
|
|
|
|
|
|
|
#cpus-per-proc: %d
|
2014-03-24 15:47:29 +00:00
|
|
|
number of cpus: %d
|
2014-01-27 22:40:51 +00:00
|
|
|
map-by: %s
|
|
|
|
|
|
2014-03-24 15:47:29 +00:00
|
|
|
Please specify a mapping level that has more cpus, or else let us
|
|
|
|
|
define a default mapping that will allow multiple cpus-per-proc.
|
After a lot of pain, I've managed to resolve the problem of conflicting mapping directives caused by mismatched MCA params - i.e., where someone has one variant of an MCA param (e.g., rmaps_base_mapping_policy) in their default MCA param file, and then specifies another variant (e.g., --npernode) on the command line. I can't fully resolve the problem as there is no way to know precisely what the user meant - we can only guess which param was really intended since the MCA param system
can't apply its normal precedence rules.
So...print a big "deprecated" warning for the old params and error out if a conflict is detected. I know that isn't what people really wanted, but it's the best we
can do. If only the old style param is given, then process it after the warning.
Extend the current map-by param to add support for ppr and cpus-per-proc, adding the latter to the list of allowed modifiers using "pe=n" for processing elements/proc. Thus, you can map-by socket:pe=2,oversubscribe to map by socket, binding 2 processing elements/process, with oversubscription allowed. Or you can map-by ppr:2:socket:pe=4 to map two processes to every socket in the allocation, binding each process to 4 processing elements.
For those wondering, a processing element is defined as a hwthread if --use-hwthreads-as-cpus is given, or else as a core.
Refs trac:4117
This commit was SVN r30620.
The following Trac tickets were found above:
Ticket 4117 --> https://svn.open-mpi.org/trac/ompi/ticket/4117
2014-02-07 21:25:40 +00:00
|
|
|
#
|
|
|
|
|
[unrecognized-modifier]
|
|
|
|
|
The mapping request contains an unrecognized modifier:
|
|
|
|
|
|
|
|
|
|
Request: %s
|
|
|
|
|
|
|
|
|
|
Please check your request and try again.
|
|
|
|
|
#
|
|
|
|
|
[invalid-pattern]
|
|
|
|
|
The mapping request contains a pattern that doesn't match
|
|
|
|
|
the required syntax of #:object
|
|
|
|
|
|
|
|
|
|
Pattern: %s
|
|
|
|
|
|
|
|
|
|
Please check your request and try again.
|
2014-02-28 16:08:52 +00:00
|
|
|
#
|
|
|
|
|
[orte-rmaps-base:oversubscribed]
|
2014-03-03 16:46:37 +00:00
|
|
|
The requested number of processes exceeds the allocated
|
2014-02-28 16:08:52 +00:00
|
|
|
number of slots:
|
|
|
|
|
|
|
|
|
|
#slots: %d
|
2014-03-03 16:46:37 +00:00
|
|
|
#processes: %d
|
2014-02-28 16:08:52 +00:00
|
|
|
|
|
|
|
|
This creates an oversubscribed condition that may adversely
|
|
|
|
|
impact performance when combined with the requested binding
|
|
|
|
|
operation. We will continue, but will not bind the processes.
|
2014-02-28 17:55:53 +00:00
|
|
|
This warning can be omitted by adding the "overload-allowed"
|
|
|
|
|
qualifier to the binding policy.
|
2014-04-02 04:17:55 +00:00
|
|
|
#
|
|
|
|
|
[cannot-launch]
|
|
|
|
|
Although we were able to map your job, we are unable to launch
|
|
|
|
|
it at this time due to required resources being busy. Please
|
|
|
|
|
try again later.
|
2014-06-01 16:14:10 +00:00
|
|
|
#
|
|
|
|
|
[rmaps:no-locale]
|
|
|
|
|
The request to bind processes could not be completed due to
|
|
|
|
|
an internal error - the locale of the following process was
|
|
|
|
|
not set by the mapper code:
|
|
|
|
|
|
|
|
|
|
Process: %s
|
|
|
|
|
|
|
|
|
|
Please contact the OMPI developers for assistance. Meantime,
|
|
|
|
|
you will still be able to run your application without binding
|
|
|
|
|
by specifying "--bind-to none" on your command line.
|
2014-06-08 20:26:59 +00:00
|
|
|
#
|
|
|
|
|
[mapping-too-low-init]
|
|
|
|
|
A request for multiple cpus-per-proc was given, but a directive
|
|
|
|
|
was also give to map to an object level that cannot support that
|
|
|
|
|
directive.
|
|
|
|
|
|
|
|
|
|
Please specify a mapping level that has more than one cpu, or
|
|
|
|
|
else let us define a default mapping that will allow multiple
|
|
|
|
|
cpus-per-proc.
|
2014-11-30 11:47:34 -08:00
|
|
|
#
|
|
|
|
|
[seq:not-enough-resources]
|
|
|
|
|
A sequential map was requested, but not enough node entries
|
|
|
|
|
were given to support the requested number of processes:
|
|
|
|
|
|
|
|
|
|
Num procs: %d
|
|
|
|
|
Num nodes: %d
|
|
|
|
|
|
|
|
|
|
We cannot continue - please adjust either the number of processes
|
|
|
|
|
or provide more node locations in the file.
|
2015-03-31 11:46:48 +03:00
|
|
|
#
|
|
|
|
|
[device-not-specified]
|
|
|
|
|
The request to map processes by distance could not be completed
|
|
|
|
|
because device to map near by was not specified. Please, use
|
|
|
|
|
rmaps_dist_device mca parameter to set it.
|
Per the discussion on the telecon, change the -host behavior so we only run one instance if no slots were provided and the user didn't specify #procs to run. However, if no slots are given and the user does specify #procs, then let the number of slots default to the #found processing elements
Ensure the returned exit status is non-zero if we fail to map
If no -np is given, but either -host and/or -hostfile was given, then error out with a message telling the user that this combination is not supported.
If -np is given, and -host is given with only one instance of each host, then default the #slots to the detected #pe's and enforce oversubscription rules.
If -np is given, and -host is given with more than one instance of a given host, then set the #slots for that host to the number of times it was given and enforce oversubscription rules. Alternatively, the #slots can be specified via "-host foo:N". I therefore believe that row #7 on Jeff's spreadsheet is incorrect.
With that one correction, this now passes all the given use-cases on that spreadsheet.
Make things behave under unmanaged allocations more like their managed cousins - if the #slots is given, then no-np shall fill things up.
Fixes #1344
2016-02-10 09:17:03 -08:00
|
|
|
#
|
|
|
|
|
[num-procs-not-specified]
|
|
|
|
|
Either the -host or -hostfile options were given, but the number
|
|
|
|
|
of processes to start was omitted. This combination is not supported.
|
|
|
|
|
|
|
|
|
|
Please specify the number of processes to run and try again.
|