Present state of MPI debugger work:

* New/improved bootstrapping technique for DLLs 
 * First cut of the MPI handle debugging interface. It is still
   evolving, but the interface is getting more stable.
 * Some minor bugs were fixed in the unity topo component (brought to
   light because of the new MPI handle debugging stuff).

Fixes trac:1209.

This commit was SVN r17730.

The following Trac tickets were found above:
  Ticket 1209 --> https://svn.open-mpi.org/trac/ompi/ticket/1209

ompi/communicator/comm.c

@@ -1446,15 +1446,14 @@ int ompi_topo_create (ompi_communicator_t *old_comm,
          * it as they deem fit */
 
         new_comm->c_topo_comm->mtc_periods_or_edges = (int *)
-        malloc (sizeof(int) * dims_or_index[ndims_or_nnodes - 1]);
+        malloc (sizeof(int) * ndims_or_nnodes);
         if (NULL == new_comm->c_topo_comm->mtc_periods_or_edges) {
             ompi_comm_free (&new_comm);
             *comm_topo = new_comm;
             return OMPI_ERROR;
         }
         memcpy (new_comm->c_topo_comm->mtc_periods_or_edges,
-                periods_or_edges, 
+                periods_or_edges, ndims_or_nnodes * sizeof(int));
-                dims_or_index[ndims_or_nnodes - 1] * sizeof(int));
 
         new_comm->c_topo_comm->mtc_coords = (int *)malloc (sizeof(int) * ndims_or_nnodes);
         if (NULL == new_comm->c_topo_comm->mtc_coords) {
@@ -1561,8 +1560,6 @@ int ompi_topo_create (ompi_communicator_t *old_comm,
     }
 
     
-    /* finally, set the communicator to comm_cart */
-
     /* if the returned rank is -1, then this process is not in the 
      * new topology, so free everything we have allocated and return */
     if (MPI_UNDEFINED == new_rank) {

ompi/debuggers/MPI_Handles_interface.txt

@@ -0,0 +1,663 @@
+1. George questions the need for the mpidbg_process_t structure (and
+   the mapping that it provides to MPI_COMM_WORLD).  What does Allinea
+   think of this?
+
+2. 3 Aug 2007: Random notes:
+   - I kept the prefix MPIDBG/mpidbg (vs. converting to mqs to be the
+     same as the message queue functionality) to allow this
+     functionality to be in a different DLL/plugin than the message
+     queue DLL/plugin.
+   - I therefore also kept our own MPIDBG return codes (the existing
+     mqs_* return codes are not sufficient).
+
+3. Some additional open issues throughout the text and header file are
+   marked with "JMS".
+
+4. 30 Aug 2007: Added questions about uninitialized / already-freed
+   handles.
+
+***************************************************************************
+
+Premise
+=======
+
+Debuggers can display the value of intrinsic datatypes (e.g., int,
+double, etc.).  Even composite struct instances can be displayed.  A
+logical extension to this concept is that the debugger should be able
+to show information about MPI opaque handles in a running application
+(and possibly cache such values for playback when the application is
+not running, such as during corefile analysis).  Similar in spirit to
+the API for obtaining the message passing queues, a simple API can be
+used between the debugger and the MPI application to obtain
+information about specific MPI handle instances.
+
+***************************************************************************
+
+Background
+==========
+
+MPI defines several types of opaque handles that are used in
+applications (e.g., MPI_Comm, MPI_Request, etc.).  The opaque handles
+are references to underlying MPI objects that are private to the MPI
+implementation.  These objects contain a wealth of information that
+can be valuable to display in a debugging context.
+
+Implementations typically have a different underlying type and then
+typedef the MPI-specified name to the underlying type.  For example,
+Open MPI has the following in mpi.h:
+
+    typedef struct ompi_communicator_t *MPI_Comm;
+
+The MPI-specified type is "MPI_Comm"; the OMPI-specific type is
+"struct ompi_communicator_t *".  The debugger cannot simply deduce the
+"real" types of MPI handles by looking at the types of well-known
+global variables (such as MPI_COMM_WORLD) because these names may be
+preprocessor macros in mpi.h; the real name of the symbol may be
+implementation-dependent and not easy to guess.
+
+Hence, if "MPI_*" types are unable to be found by the debugger within
+the application image, the MPI implementation will need to provide a
+list of the actual types to the debugger when the MPI handle
+interpretation functionality is initialized.  Once the debugger knows
+the types of MPI handles, it can provide context-sensitive information
+when displaying the values of variables within the application image.
+
+Some MPI implementations use integers for handles in C.  Such
+implementations are strongly encouraged to use "typedef" for creating
+handle types in mpi.h (vs. using #define), such as:
+
+    typedef int MPI_Comm;
+    typedef int MPI_Datatype;
+    /* etc. */
+
+So that the debugger can identify a variable as an MPI_Comm (vs. an
+int) and therefore know that it is an MPI communicator.
+
+In Fortran, however, all MPI handles are defined by the MPI standard
+to be of type INTEGER.  As such, there is no way for the debugger to
+automatically know that a given INTEGER variable is an MPI handle, nor
+which kind of MPI handle.  It is therefore up to the debugger's UI to
+allow users to designate specific INTEGER variables as a given MPI
+handle type.
+
+MPI handles can be "freed" by the application, but this actually only
+marks the underlying MPI object for freeing; the object itself may not
+be freed until all corresponding handles and pending actions have
+completed.  Additionally, for debugging purposes, an MPI
+implementation can choose to *never* free MPI objects (in order to
+show that they were marked as freed and/or actually freed).
+
+***************************************************************************
+
+Assumptions
+===========
+
+Some terminology:
+ - host: machine and process on which debugging process is executing.
+ - debugger: machine and debugging UI, where the debugger is running.
+   The two machines may be distinct from a hardware point of view,
+   they may have differing endinanness, word-size, etc.
+
+MPI typically denotes function names in all capitol letters (MPI_INIT)
+as a language-neutral form.  The Fortran binding for the function is
+dependent upon the compiler; the C binding for the function
+capitolizes the "MPI" and the first letter of the next token (e.g.,
+"MPI_Init").  This text will use the language-neutral names for all
+MPI function names.
+
+The debugger will access the handle-interpretation functionality by
+loading a plugin provided by the MPI implementation into its process
+space.  The plugin will contain "query" functions that the debugger
+can invoke to obtain information about various MPI handle types.  The
+query functions generally return the additional information or "not
+found" kinds of errors.
+
+The MPI-implementation-provided plugin shares many of the same
+characteristics as the Etnus MPI message queue plugin design, but is
+loaded slightly differently.  The plugin will use the mqs_* functions
+defined by the Etnus message queue access interface to read the
+process image to obtain MPI object information that is then passed
+back to the debugger.
+
+The plugin's query functions should not be called before MPI_INIT has
+completed nor after MPI_FINALIZE has started.  MPI handles are only
+meaningful between the time that MPI_INIT completes and MPI_FINALIZE
+starts, anyway.
+
+When MPI handles are marked for freeing by the MPI implementation,
+there should be some notice from the debugger that the underlying
+objects should not *actually* be freed, but rather orphaned.  The
+debugger can track these objects and keep a reference count of
+how many handles are still referring to the underlying objects.  When
+the reference count goes to 0, the debugger can call a function in the
+application telling the MPI implementation that the object is safe to
+be freed.
+
+In this way, valuable debugging information is still available to the
+user if they have stale MPI handles because the underlying object will
+still be available in memory (marked as "stale"); but MPI object
+memory usage is not cumulative.
+
+The debugger may not be able to provide any additional information in
+Fortran applications because all MPI handles are of type INTEGER (and
+there's no way to tell that any given integer is an MPI communicator
+handle, for example) unless the debugger provides some way in the UI
+to indicate that a given INTEGER variable is a specific type of MPI
+handle.
+
+Note that the following pattern is valid in MPI:
+
+  MPI_Request a, b;
+  MPI_Isend(..., &a);
+  b = a;
+  MPI_Request_free(&a);
+
+After executing MPI_REQUEST_FREE, the handle "a" has been set to
+MPI_REQUEST_NULL, but the handle "b" still points to the [potentially
+ongoing] request.  "b" would therefore report that it has been marked
+for freeing by the application, but would not report that it was
+completed / marked for freeing by the MPI implementation until the
+corresponding ISEND actually completes.
+
+The query functions will return newly-allocated structs of information
+to the debugger (allocated via mqs_malloc).  The debugger will be
+responsible for freeing this memory.  Arrays/lists of information
+contained in the structs must be individually freed if they are not
+NULL (i.e., they will each be allocated via mqs_malloc).
+
+Finally, note that not all of this needs to be supported by the
+debugger at once.  Interpretation of some of the more common handle
+types can be implemented first (e.g., communicators and requests),
+followed by more types over time.
+
+***************************************************************************
+
+MPI handle types
+================
+
+Communicator
+============
+
+C: MPI_Comm
+C++: MPI::Comm, MPI::Intracomm, MPI::Intercomm, MPI::Cartcomm,
+     MPI::Graphcomm
+
+A communicator is an ordered set of MPI processes and a unique
+communication context.  There are 3 predefined communicators
+(MPI_COMM_WORLD, MPI_COMM_SELF, MPI_COMM_PARENT), and applications can
+create their own communicators.  There are two types of communicators:
+
+1. Intracommunicator: contains one group of processes.
+   Intracommunicators may optionally have a topology:
+   - Cartesian: a dense, N-dimensional Cartesian topology
+   - Graph: an arbitrary unweighted, undirected, connected graph
+
+2. Intercommunicator: contains a local group and a remote group of
+   processes.
+
+--> Information available from communicators:
+
+  - String name of length MPI_MAX_OBJECT_NAME
+  - Flag indicating whether it's a predefined communicator or not
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - This process' unique ID within the communicator ("rank", from 0-(N-1))
+  - This process' unique ID in the entire MPI universe
+  - Number of peer processes in the communicator
+  - Type of communicator: inter or intra
+    - If Inter, the number and list of peer processes in the communicator
+    - If intra, whether the communicator has a graph or cartesian
+      topology
+    - If have a Cartesian toplogy (mutually exclusive with having a
+      graph topology), the topology information:
+      - "ndims", "dims", periods" arguments from the corresponding
+        call to MPI_CART_CREATE, describing the Cartesian topology.
+    - If have a graph topology (mutually exclusive with having a
+      cartesian topology):
+      - "index" and "edges" arguments from the corresponding call to
+        MPI_GRAPH_CREATE, describing the nodes and edges in the graph
+        topology
+  - Cross reference to the underlying MPI group(s)
+  - Cross reference to the underlying MPI error handler
+
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - A list of MPI "attributes" that are attached to the communicator
+  - A list of any MPI requests currently associated with the
+    communicator (e.g., ongoing or completed but not released
+    communicator requests, or, in multi-threaded scenarios, ongoing
+    communicator operations potentially from other threads)
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - A list of MPI windows using this communicator
+  - A list of MPI files using this communicator
+
+--> Suggested data types
+    See mpihandles_interface.h
+
+--> Suggested API functions
+    See mpihandles_interface.h
+
+---------------------------------------------------------------------------
+
+Datatype
+========
+
+C: MPI_Datatype
+C++: MPI::Datatype
+
+MPI datatypes are used to express the size and shape of user-defind
+messages.  For example, a user can define an MPI datatype that
+describes a C structure, and can then use that MPI datatype to send
+and receive instances (or arrays) of that struct.  There are many
+predefined datatypes, and applications can create their own datatypes.
+
+--> Information available from datatypes:
+
+  - String name
+  - Flag indicating whether it's a predefined datatype or not
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - A type map of the overall datatype, composed of *only* intrinsic MPI
+    datatypes (MPI_INT, MPI_DOUBLE, etc.) that can be rendered by the
+    debugger into a form similar to MPI-1:3.12
+  - What function was used to create the datatype:
+    - <MPI_TYPE_>CREATE_DARRAY, CREATE_F90_COMPLEX,
+      CREATE_F90_INTEGER, CREATE_F90_REAL, CREATE_HINDEXED,
+      CREATE_HVACTOR, CREATE_INDEXED_BLOCK, CREATE_RESIZED,
+      CREATE_STRUCT, CREATE_SUBARRAY
+      --> JMS: Do we need to differentiate between the MPI-1 and MPI-2
+          functions?  Probably worthwhile, if for nothing other than
+          completeness (e.g., don't confuse the user saying that a
+          datatype was created by MPI_TYPE_CREATE_STRUCT when it was
+          creally created with MPI_TYPE_STRUCT, even though they're
+          effectively equivalent).
+    - <MPI_>TYPE_HINDEXED, TYPE_INDEXED, TYPE_HVECTOR, TYPE_VECTOR,
+      TYPE_STRUCT, TYPE_CONTIGUOUS, 
+
+  JMS: with the type map provided by MPI, a debugger can show "holes"
+       in a datatype (potentially indicating missed optimizations by
+       the application).  Very cool/useful!
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - Ongoing communication actions involving the datatype
+    (point-to-point, collective, one-sided)
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - Whether the datatype has been "committed" or not
+  - A list of datatypes used to create this datatype (JMS: may
+    require caching by the debugger!!)
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Error handler
+=============
+
+C: MPI_Errhandler
+C++: MPI::Errhandler
+
+MPI allows applications to define their own error handlers.  The
+default error handler is to abort the MPI job.  Error handlers can be
+attached to communicators, files, and windows.  There are 3 predefined
+error handlers (MPI_ERRORS_ARE_FATAL, MPI_ERRORS_RETURN,
+MPI::ERRORS_THROW_EXCEPTIONS), and applications can create their own
+error handlers.
+
+--> Information available from error handlers:
+
+  - Flag indicating whether it's a predefined error handler or not
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - Type of errorhandler: communicator, file, window
+  - If user-defined (i.e., not predefined), the function pointer for
+    the user function that MPI will invoke upon error
+  - Whether the callback function is in Fortran or C/C++
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - String name for predefined handles
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - List of communicators/files/windows that this error handler is
+    currently attached to
+
+--> Suggested data types
+    See mpihandles_interface.h
+
+--> Suggested API functions
+    See mpihandles_interface.h
+
+---------------------------------------------------------------------------
+
+File
+====
+
+C: MPI_File
+C++: MPI::File
+
+MPI has the concept of parallel IO, where a group of processes
+collectively open, read/write, and close files.  An MPI_File handle
+represents both an ordered set of processes and a file to which they
+are accessing.  There is one pre-defined file: MPI_FILE_NULL;
+applications can open their own files.
+
+--> Information available from files:
+
+  - String file name (or "MPI_FILE_NULL")
+  - Flag indicating whether it's a predefined file or not
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - Communicator that the file was opened with
+  - Info key=value pairs that the file was opened with
+  - Mode that the file was opened with
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - Whether the handle has been marked for freeing (closing) by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - A list of any MPI requests currently associated with the file
+    (e.g., ongoing or completed but not released file requests, or, in
+    multi-threaded scenarios, ongoing file operations potentially from
+    other threads)
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Group
+=====
+
+C: MPI_Group
+C++: MPI::Group
+
+An unordered set of processes.  There are predefined and user-defined
+groups.  Every communicator contains exactly 1 or 2 groups (depending
+on the type of communicator).  There are 2 predefined groups
+(MPI_GROUP_NULL and MPI_GROUP_EMPTY); applications can create their
+own groups.
+
+--> Information available from groups:
+
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - This process' unique ID in this group
+  - List of peer processes in this group
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - String name for predefined handles
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - A list of MPI communicators using this group
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Info
+====
+
+C: MPI_Info
+C++: MPI::Info
+
+A set of key=value pairs (the key and value are separate strings) that
+can be used to pass "hints" to MPI.  There are no predefined info
+handles; applications can create their own info handles.
+
+--> Information available from info:
+
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - Number of key=value pairs on the info
+  - List of key=value pairs (each key and value is an individual
+    string)
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - A list of places where the info object is currently being used
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Request
+=======
+
+C: MPI_Request
+C++:: MPI::Request, MPI::Grequest, MPI::Prequest
+
+A pointer to an ongoing or completed-but-not-yet-released action.
+There are three types of requests:
+
+  - Point-to-point communication: non-blocking sends, receives
+  - File actions: non-blocking reads, writes
+  - Generalized actions: Users can define their own asynchronous actions
+    that can be subject to MPI completion semantics
+
+There is one predefined request (MPI_REQUEST_NULL); applications can
+create their own requests.
+
+--> Information available from requests:
+
+  - Flag indicating whether it's a predefined request or not
+  - Flag indicating whether the request is persistent or not
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - Type of the request: pt2pt, file, generalized
+  - Function that created this request:
+    - Pt2pt: <MPI_>ISEND, IBSEND, ISSEND, IRSEND, IRECV, SEND_INIT,
+             BSEND_INIT, SSEND_INIT, RSEND_INIT, RECV_INIT
+    - File: <MPI_FILE_>IREAD, IREAD_AT, IREAD_SHARED, IWRITE,
+            IWRITE_AT, IWRITE_SHARED
+  - Whether the request has been marked "complete" or not
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - String name for predefined handles
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - Peer process(es) invovled with the request (if available)
+  - If pt2pt, communicator associated with the request
+  - If file, file associated with the request
+  - If pt2pt or file, whether the data transfer has started yet
+  - If pt2pt or file, whether the data transfer has completed yet
+
+--> Suggested data types
+    See mpihandles_interface.h
+
+--> Suggested API functions
+    See mpihandles_interface.h
+
+---------------------------------------------------------------------------
+
+Operation
+=========
+
+C: MPI_Op
+C++:: MPI::Op
+
+A reduction operator used in MPI collective and one-sided operations
+(e.g., sum, multiply, etc.).  There are several predefined operators;
+applications can also create their own operators.
+
+--> Information available from operators:
+
+  - Flag indicating whether it's a predefined operator or not
+  - C handle (if available)
+  - Fortran integer index for this handle (if available)
+  - If user-defined, the function pointer for the user function that
+    MPI will invoke
+  - Whether the callback function is in Fortran or C/C++
+  - Whether the operator is commutative or not
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - String name for predefined handles
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - List of ongoing collective / one-sided communications associated
+    with this operator
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Status
+======
+
+C: MPI_Status
+C++: MPI::Status
+
+A user-accessible struct that contains information about a completed
+communication.  The MPI status is a little different from other MPI
+handles in that it is the object itselt; not a handle to an underlying
+MPI status.  For example, if a point-to-point communication was
+started with a wildcard receive, the status will contain information
+about the peer to whom the communication completed.  There are no
+predefined statuses.
+
+--> Information available from status:
+
+  - Public member MPI_SOURCE: source of the communication
+  - Public member MPI_TAG: tag of the communication
+  - Public member MPI_ERROR: error status of the communication
+  - Number of bytes in the communication
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - Number of data elements in the communication
+
+--> Suggested data types
+    See mpihandles_interface.h
+
+--> Suggested API functions
+    See mpihandles_interface.h
+
+---------------------------------------------------------------------------
+
+Window
+======
+
+C: MPI_Win
+C++: MPI::Win
+
+An ordered set of processes, each defining their own "window" of
+memory for one-sided operations.
+
+--> Information available from windows:
+
+  - Communicator that the window was created with
+  - Base address, length, and displacement units of the window *in this
+    process*
+  - Info key=value pairs that the file was opened with
+
+
+--> Extra/bonus information that the MPI may be able to provide:
+
+  - Whether the handle has been marked for freeing by the
+    application or not
+  - Whether the underlying object has been "freed" by the MPI
+    implementation (i.e., made inactive, and would have actually been
+    freed if not running under a debugger)
+  - Whether LOCK has been called on this window without a
+    corresponding UNLOCK yet
+  - Whether START has been called on this window without a
+    corresponding COMPLETE yet
+  - Whether POST has been called on this window without a
+    corresopnding TEST/WAIT yet
+  - What the last synchronization call was on the window: FENCE,
+    LOCK/UNLOCK, START/COMPLETE, POST/TEST/WAIT
+
+--> Suggested data types
+
+    ***TO BE FILLED IN***
+
+--> Suggested API functions
+
+    ***TO BE FILLED IN***
+
+---------------------------------------------------------------------------
+
+Address integer
+===============
+
+C: MPI_Aint
+C++: MPI::Aint
+
+This is an MPI-specific type, but is always an integer value that is
+large enough to hold addresses.  It is typically an 32 or 64 bits
+long.  Hence, the debugger should be able to directly display this
+value.
+
+---------------------------------------------------------------------------
+
+Offset
+======
+
+C: MPI_Offset
+C++: MPI::Offset
+
+This is a MPI-specific type, but is always an integer value that is
+large enough to hold file offsets.  It is typically an 32 or 64 bits
+long.  Hence, the debugger should be able to directly display this
+value.

ompi/debuggers/Makefile.am

@@ -9,6 +9,7 @@
 #                         University of Stuttgart.  All rights reserved.
 # Copyright (c) 2004-2005 The Regents of the University of California.
 #                         All rights reserved.
+# Copyright (c) 2007      Cisco Systems, Inc.  All rights reserved.
 # $COPYRIGHT$
 # 
 # Additional copyrights may follow
@@ -17,7 +18,7 @@
 #
 
 noinst_LTLIBRARIES = libdebuggers.la
-inst_LTLIBRARIES = libompitv.la
+pkglib_LTLIBRARIES = libompi_dbg_msgq.la
 
 # This is not quite in the Automake spirit, but we have to do it.
 # Since the totalview portion of the library must be built with -g, we
@@ -30,18 +31,25 @@ CFLAGS = $(CFLAGS_WITHOUT_OPTFLAGS) $(TOTALVIEW_DEBUG_FLAGS)
 
 # Source code files
 
-headers = debuggers.h mpi_interface.h ompi_dll_defs.h
+headers = \
-
+        debuggers.h \
-instdir = $(libdir)/openmpi
+        ompi_common_dll_defs.h \
+        msgq_interface.h ompi_msgq_dll_defs.h
 
 libdebuggers_la_SOURCES = \
         $(headers) \
-        ompi_totalview.c
+        ompi_debuggers.c
-libdebuggers_la_CFLAGS = -DOMPI_TV_DLL=\"$(instdir)/libompitv.so\"
+libdebuggers_la_CPPFLAGS = \
+        -DOMPI_MSGQ_DLL=\"$(pkglibdir)/libompi_dbg_msgq.so\" \
+        -DOMPI_MSGQ_DLL_PREFIX=\"libompi_dbg_msgq\" \
+        -DOMPI_MPIHANDLES_DLL_PREFIX=\"libompi_dbg_mpihandles\"
 
-libompitv_la_SOURCES = ompi_dll.c
+common = ompi_common_dll_defs.h ompi_common_dll.c
-libompitv_la_CFLAGS = -g
+
-libompitv_la_LDFLAGS = -module -avoid-version
+# MPI message queue DLL
+libompi_dbg_msgq_la_SOURCES = ompi_msgq_dll.c ompi_msgq_dll_defs.h $(common)
+libompi_dbg_msgq_la_CFLAGS = -g
+libompi_dbg_msgq_la_LDFLAGS = -module -avoid-version
 
 # Conditionally install the header files
 

ompi/debuggers/debuggers.h

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2007      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -27,16 +28,13 @@
 
 #include "ompi_config.h"
 
-#if defined(c_plusplus) || defined(__cplusplus)
+BEGIN_C_DECLS
-extern "C" {
-#endif
-    /**
-     * Wait for a TotalView-like debugger if asked.
-     */
-    OMPI_DECLSPEC void ompi_wait_for_totalview(void);
 
-#if defined(c_plusplus) || defined(__cplusplus)
+    /**
-}
+     * Wait for a debugger if asked.
-#endif
+     */
+    OMPI_DECLSPEC void ompi_wait_for_debugger(void);
+
+END_C_DECLS
 
 #endif /* OMPI_DEBUGGERS_H */

ompi/debuggers/mpihandles_interface.h

@@ -0,0 +1,876 @@
+/*
+ * Copyright (c) 2007      High Performance Computing Center Stuttgart, 
+ *                         University of Stuttgart.  All rights reserved.
+ * Copyright (c) 2007-2008 Cisco, Inc.  All rights reserved.
+ * Copyright (c) 2007      The University of Tennessee and The University of
+ *                         Tennessee Research Foundation.  All rights reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * Some text copied from and references made to mpi_interface.h.
+ * 
+ * Copyright (C) 2000-2004 by Etnus, LLC
+ * Copyright (C) 1999 by Etnus, Inc.
+ * Copyright (C) 1997-1998 Dolphin Interconnect Solutions Inc.
+ *
+ * $HEADER$
+ */
+
+#ifndef __MPIDBG_INTERFACE_H__
+#define __MPIDBG_INTERFACE_H__ 1
+
+/*
+ * This file provides interface functions for a debugger to gather
+ * additional information about MPI handles.
+ */
+#include <sys/types.h>
+
+/* Include the Etnus debugger message queue interface so that we can
+   use much of its infrastructure (e.g., the mqs_basic_callbacks,
+   mqs_image_callbacks, and mqs_process_callbacks). */
+#define FOR_MPI2 0
+#include "msgq_interface.h"
+
+/**************************************************************************
+ * Types and macros
+ **************************************************************************/
+
+enum {
+    MPIDBG_MAX_OBJECT_NAME = 64
+};
+enum {
+    MPIDBG_MAX_FILENAME = 1024
+};
+enum {
+    MPIDBG_INTERFACE_VERSION = 1
+};
+
+
+/*-----------------------------------------------------------------------
+ * Global initialization information for the DLL
+ *-----------------------------------------------------------------------*/
+
+/* Structure containing types for C and C++ MPI handles */
+struct mpidbg_handle_info_t {
+    /* C handle types.  They are typically pointers to something or
+       integers. */
+    /* Back-end type for MPI_Aint */
+    mqs_type *hi_c_aint;
+    /* Back-end type for MPI_Comm */
+    mqs_type *hi_c_comm;
+    /* Back-end type for MPI_Datatype */
+    mqs_type *hi_c_datatype;
+    /* Back-end type for MPI_Errhandler */
+    mqs_type *hi_c_errhandler;
+    /* Back-end type for MPI_File */
+    mqs_type *hi_c_file;
+    /* Back-end type for MPI_Group */
+    mqs_type *hi_c_group;
+    /* Back-end type for MPI_Info */
+    mqs_type *hi_c_info;
+    /* Back-end type for MPI_Offset */
+    mqs_type *hi_c_offset;
+    /* Back-end type for MPI_Op */
+    mqs_type *hi_c_op;
+    /* Back-end type for MPI_Request */
+    mqs_type *hi_c_request;
+    /* Back-end type for MPI_Status */
+    mqs_type *hi_c_status;
+    /* Back-end type for MPI_Win */
+    mqs_type *hi_c_win;
+
+    /* C++ handle types.  Note that these will always be *objects*,
+       never pointers. */
+    /* Back-end type for MPI::Aint */
+    mqs_type *hi_cxx_aint;
+    /* Back-end type for MPI::Comm */
+    mqs_type *hi_cxx_comm;
+    /* Back-end type for MPI::Intracomm */
+    mqs_type *hi_cxx_intracomm;
+    /* Back-end type for MPI::Intercomm */
+    mqs_type *hi_cxx_intercomm;
+    /* Back-end type for MPI::Graphcomm */
+    mqs_type *hi_cxx_graphcomm;
+    /* Back-end type for MPI::Cartcomm */
+    mqs_type *hi_cxx_cartcomm;
+    /* Back-end type for MPI::Datatype */
+    mqs_type *hi_cxx_datatype;
+    /* Back-end type for MPI::Errhandler */
+    mqs_type *hi_cxx_errhandler;
+    /* Back-end type for MPI::File */
+    mqs_type *hi_cxx_file;
+    /* Back-end type for MPI::Group */
+    mqs_type *hi_cxx_group;
+    /* Back-end type for MPI::Info */
+    mqs_type *hi_cxx_info;
+    /* Back-end type for MPI::Offset */
+    mqs_type *hi_cxx_offset;
+    /* Back-end type for MPI::Op */
+    mqs_type *hi_cxx_op;
+    /* Back-end type for MPI::Request */
+    mqs_type *hi_cxx_request;
+    /* Back-end type for MPI::Prequest */
+    mqs_type *hi_cxx_prequest;
+    /* Back-end type for MPI::Grequest */
+    mqs_type *hi_cxx_grequest;
+    /* Back-end type for MPI::Status */
+    mqs_type *hi_cxx_status;
+    /* Back-end type for MPI::Win */
+    mqs_type *hi_cxx_win;
+};
+
+enum mpidbg_return_codes_t {
+    /* Success */
+    MPIDBG_SUCCESS,
+    /* Something was not found */
+    MPIDBG_ERR_NOT_FOUND,
+    /* Something is not supported */
+    MPIDBG_ERR_NOT_SUPPORTED,
+    /* Something is out of range */
+    MPIDBG_ERR_OUT_OF_RANGE,
+    /* Something is not available */
+    MPIDBG_ERR_UNAVAILABLE,
+    /* Ran out of memory */
+    MPIDBG_ERR_NO_MEM,
+    /* Sentinel max value */
+    MPIDBG_MAX_RETURN_CODE
+};
+
+/*-----------------------------------------------------------------------
+ * General data structures
+ *-----------------------------------------------------------------------*/
+
+/* Information about MPI processes */
+struct mpidbg_process_t {
+    /* JMS: need something to uniquely ID MPI processes in the
+       presence of MPI_COMM_SPAWN */
+
+    /* Global rank in MPI_COMM_WORLD */
+    int mpi_comm_world_rank;
+};
+/* ==> JMS Should we just use mqs_process_location instead?  George
+   thinks that this is unncessary -- perhaps due to the fact that we
+   could use mqs_process_location...?  Need to get some feedback from
+   Allinea on this one. */
+
+/* General name -> handle address mappings.  This is an optional type
+   that is used to describe MPI's predefined handles if the
+   pre-defined names do not appear as symbols in the MPI process.
+   E.g., if MPI_COMM_WORLD is a #define that maps to some other value,
+   this data structure can be used to map the string "MPI_COMM_WORLD"
+   to the actual value of the handle that it corresponds to (e.g., 0
+   or a pointer value). */
+struct mpidbg_name_map_t {
+    /* Name of the handle */
+    char *map_name;
+
+    /* Handle that the name corresponds to.  Will be 0/NULL if there
+       is no corresponding back-end object. */
+    mqs_taddr_t map_handle;
+};
+
+/* MPI attribute / value pairs.  Include both a numeric and string
+   key; pre-defined MPI keyvals (e.g., MPI_TAG_MAX) have a
+   human-readable string name.  The string will be NULL for
+   non-predefined keyvals. */
+struct mpidbg_attribute_pair_t {
+    /* Keyval */
+    int keyval;
+    /* Keyval name; will be non-NULL for attributes that have a
+       human-readable name (e.g., MPI predefined keyvals) */
+    char *keyval_name;
+    /* Value */
+    char *value;
+};
+
+/*-----------------------------------------------------------------------
+ * Communicators
+ *-----------------------------------------------------------------------*/
+
+/* Using an enum instead of #define because debuggers can show the
+   *names* of enum values, not just the values. */
+enum mpidbg_comm_capabilities_t {
+    /* Whether this MPI DLL supports returning basic information about
+       communicators */
+    MPIDBG_COMM_CAP_BASIC =                  0x01,
+    /* Whether this MPI DLL supports returning names of
+       communicators */
+    MPIDBG_COMM_CAP_STRING_NAMES =           0x02,
+    /* Whether this MPI DLL supports indicating whether a communicator
+       has been freed by the user application */
+    MPIDBG_COMM_CAP_FREED_HANDLE =           0x04,
+    /* Whether this MPI DLL supports indicating whether a communicator
+       object has been freed by the MPI implementation or not */
+    MPIDBG_COMM_CAP_FREED_OBJECT =           0x08,
+    /* Whether this MPI DLL supports returning the list of MPI request
+       handles that are pending on a communicator */
+    MPIDBG_COMM_CAP_REQUEST_LIST =           0x10,
+    /* Whether this MPI DLL supports returning the list of MPI window
+       handles that were derived from a given communicator */
+    MPIDBG_COMM_CAP_WINDOW_LIST =            0x20,
+    /* Whether this MPI DLL supports returning the list of MPI file
+       handles that were derived from a given communicator */
+    MPIDBG_COMM_CAP_FILE_LIST =              0x40,
+    /* Sentinel max value */
+    MPIDBG_COMM_CAP_MAX
+};
+
+enum mpidbg_comm_info_bitmap_t {
+    /* Predefined communicator if set (user-defined if not set) */
+    MPIDBG_COMM_INFO_PREDEFINED =      0x01,
+    /* Whether this communicator is a cartesian communicator or not
+       (mutually exclusive with _GRAPH and _INTERCOMM) */
+    MPIDBG_COMM_INFO_CARTESIAN =       0x02,
+    /* Whether this communicator is a graph communicator or not
+       (mutually exclusive with _CARTESIAN and _INTERCOMM) */
+    MPIDBG_COMM_INFO_GRAPH =           0x04,
+    /* If a cartesian or graph communicator, whether the processes in
+       this communicator were re-ordered when the topology was
+       assigned. */
+    MPIDBG_COMM_INFO_TOPO_REORDERED =  0x08,
+    /* Whether this is an intercommunicator or not (this communicator
+       is an intracommunicator if this flag is not yet). */
+    MPIDBG_COMM_INFO_INTERCOMM =       0x10,
+    /* This communicator has been marked for freeing by the user
+       application if set */
+    MPIDBG_COMM_INFO_FREED_HANDLE =    0x20,
+    /* This communicator has actually been freed by the MPI
+       implementation if set */
+    MPIDBG_COMM_INFO_FREED_OBJECT =    0x40,
+    /* The queried communicator is MPI_COMM_NULL */
+    MPIDBG_COMM_INFO_COMM_NULL =       0x80,
+    /* Sentinel max value */
+    MPIDBG_COMM_INFO_MAX
+};
+
+struct mpidbg_comm_info_t {
+    /* Name of the MPI_COMM */
+    char comm_name[MPIDBG_MAX_OBJECT_NAME];
+
+    /* Bit flags describing the communicator */
+    enum mpidbg_comm_info_bitmap_t comm_bitflags;
+
+    /* This process' rank within this communicator */
+    int comm_rank;
+    /* The communicator's size  */
+    int comm_size;
+
+    /* Number of processes in the local group */
+    int comm_num_local_procs;
+    /* Information about each process in the local group (in
+       communicator rank order, length: comm_num_local_procs) */
+    struct mpidbg_process_t *comm_local_procs;
+
+    /* For intercommunicators, the number of processes in the remote
+       group */
+    int comm_num_remote_procs;
+    /* For intercommunicators, information about each process in the
+       remote group (in communicator rank order, length:
+       comm_num_remote_procs) */
+    struct mpidbg_process_t *comm_remote_procs;
+
+    /* For cartesian communicators, the number of dimensions */
+    int comm_cart_num_dims;
+    /* For cartesian communicators, an array of dimension lengths
+       (length: cart_comm_num_dims) */
+    int *comm_cart_dims;
+    /* For cartesian communicators, an array of boolean values
+       indicating whether each dimension is periodic or not (length:
+       cart_comm_num_dims) */
+    int8_t *comm_cart_periods;
+
+    /* For graph communicators, the number of nodes */
+    int comm_graph_num_nodes;
+    /* For graph communicators, an array of the node degrees (length:
+       comm_graph_num_nodes) */
+    int *comm_graph_index;
+    /* For graph communicators, an array of the edges (length:
+       comm_graph_num_nodes) */
+    int *comm_graph_edges;
+
+    /* C handle */
+    mqs_taddr_t comm_c_handle;
+    /* Fortran handle; will be MPIDBG_ERR_UNAVAILABLE if currently
+       unavailable or MPIDBG_ERR_NOT_SUPPORTED if not supported */
+    int comm_fortran_handle;
+
+    /* Number of attributes defined on this communicator */
+    int comm_num_attrs;
+    /* Array of attribute keyval/value pairs defined on this
+       communicator (length: comm_num_attrs) */
+    struct mpidbg_attribute_pair_t *comm_attrs;
+
+    /* Number of ongoing requests within this communicator, or
+       MPIDBG_ERR_NOT_SUPPORTED */
+    int comm_num_pending_requests;
+    /* If comm_num_pending_requests != MPIDBG_ERR_NOT_SUPPORTED, an
+       array of ongoing request handles attached on this
+       communicator (length: comm_num_pending_requests) */
+    mqs_taddr_t *comm_pending_requests;
+
+    /* Number of MPI windows derived from this communicator, or
+       MPIDBG_ERR_NOT_SUPPORTED  */
+    int comm_num_derived_windows;
+    /* If comm_num_derived_windows != MPIDBG_ERR_NOT_SUPPORTED, an
+       array of window handles derived from this communicator (length:
+       com_num_derived_windows) */
+    mqs_taddr_t *comm_derived_windows;
+
+    /* Number of MPI files derived from this communicator, or
+       MPIDBG_ERR_NOT_SUPPORTED  */
+    int comm_num_derived_files;
+    /* If comm_num_derived_files != MPIDBG_ERR_NOT_SUPPORTED, an array
+       of file handles derived from this communicator (length:
+       comm_num_derived_files) */
+    mqs_taddr_t *comm_derived_files;
+};
+
+
+/*-----------------------------------------------------------------------
+ * Requests
+ *-----------------------------------------------------------------------*/
+
+/* Using an enum instead of #define because debuggers can show the
+   *names* of enum values, not just the values. */
+enum mpidbg_request_capabilities_t {
+    /* Whether this MPI DLL supports returning basic information about
+       requests */
+    MPIDBG_REQUEST_CAP_BASIC =           0x01,
+    /* Sentinel max value */
+    MPIDBG_REQUEST_CAP_MAX
+};
+
+enum mpidbg_request_info_bitmap_t {
+    /* Predefined request if set (user-defined if not set) */
+    MPIDBG_REQUEST_INFO_PREDEFINED =      0x01,
+    /* Sentinel max value */
+    MPIDBG_REQUEST_INFO_MAX
+};
+
+struct mpidbg_request_info_t {
+    /* Bit flags describing the error handler */
+    enum mpidbg_request_info_bitmap_t req_bitflags;
+
+    /* C handle */
+    mqs_taddr_t req_c_handle;
+    /* Fortran handle; will be MPIDBG_ERR_UNAVAILABLE if currently
+       unavailable or MPIDBG_ERR_NOT_SUPPORTED if not supported */
+    int req_fortran_handle;
+};
+
+/*-----------------------------------------------------------------------
+ * Statuses
+ *-----------------------------------------------------------------------*/
+
+enum mpidbg_status_capabilities_t {
+    /* Whether this MPI DLL supports returning basic information about
+       statuses */
+    MPIDBG_STATUS_CAP_BASIC =           0x01,
+    /* Sentinel max value */
+    MPIDBG_STATUS_CAP_MAX
+};
+
+enum mpidbg_status_info_bitmap_t {
+    /* Predefined status if set (user-defined if not set) */
+    MPIDBG_STATUS_INFO_PREDEFINED =      0x01,
+    /* Sentinel max value */
+    MPIDBG_STATUS_INFO_MAX
+};
+
+struct mpidbg_status_info_t {
+    /* Bit flags describing the error handler */
+    enum mpidbg_status_info_bitmap_t status_bitflags;
+};
+
+/*-----------------------------------------------------------------------
+ * Error handlers
+ *-----------------------------------------------------------------------*/
+
+/* Using an enum instead of #define because debuggers can show the
+   *names* of enum values, not just the values. */
+enum mpidbg_errhandler_capabilities_t {
+    /* Whether this MPI DLL supports returning basic information about
+       error handlers */
+    MPIDBG_ERRH_CAP_BASIC =           0x01,
+    /* Whether this MPI DLL supports returning names of the predefined
+       error handlers */
+    MPIDBG_ERRH_CAP_STRING_NAMES =    0x02,
+    /* Whether this MPI DLL supports indicating whether an error
+       handler has been freed by the user application */
+    MPIDBG_ERRH_CAP_FREED_HANDLE =    0x04,
+    /* Whether this MPI DLL supports indicating whether an error
+       handler object has been freed by the MPI implementation or
+       not */
+    MPIDBG_ERRH_CAP_FREED_OBJECT =    0x08,
+    /* Whether this MPI DLL supports returning the list of MPI handles
+       that an MPI error handler is attached to */
+    MPIDBG_ERRH_CAP_HANDLE_LIST =     0x10,
+    /* Sentinel max value */
+    MPIDBG_ERRH_CAP_MAX
+};
+
+enum mpidbg_errhandler_info_bitmap_t {
+    /* Predefined error handler if set (user-defined if not set) */
+    MPIDBG_ERRH_INFO_PREDEFINED =      0x01,
+    /* Communicator error handler if set */
+    MPIDBG_ERRH_INFO_COMMUNICATOR =    0x02,
+    /* File error handler if set */
+    MPIDBG_ERRH_INFO_FILE =            0x04,
+    /* Window error handler if set */
+    MPIDBG_ERRH_INFO_WINDOW =          0x08,
+    /* Callback is in C if set (Fortran if not set) */
+    MPIDBG_ERRH_INFO_C_CALLBACK =      0x10,
+    /* This errorhandler has been marked for freeing by the user
+       application if set */
+    MPIDBG_ERRH_INFO_FREED_HANDLE =    0x20,
+    /* This errorhandler has actually been freed by the MPI
+       implementation if set */
+    MPIDBG_ERRH_INFO_FREED_OBJECT =    0x40,
+    /* Sentinel max value */
+    MPIDBG_ERRH_INFO_MAX
+};
+
+struct mpidbg_errhandler_info_t {
+    /* String name; only relevant for predefined errorhandlers.  If
+       not a predefined errorhandler, eh_name[0] will be '\0'; */
+    char eh_name[MPIDBG_MAX_OBJECT_NAME];
+
+    /* Bit flags describing the error handler */
+    enum mpidbg_errhandler_info_bitmap_t eh_bitflags;
+
+    /* C handle */
+    mqs_taddr_t eh_c_handle;
+    /* Fortran handle; will be MPIDBG_ERR_UNAVAILABLE if currently
+       unavailable or MPIDBG_ERR_NOT_SUPPORTED if not supported */
+    int eh_fortran_handle;
+
+    /* Number of MPI handles that this error handler is attached to.
+       MPIDBG_ERR_NOT_SUPPORTED means that this information is not
+       supported by the DLL. */
+    int16_t eh_refcount;
+    /* If eh_refcount != MPIDBG_ERR_NOT_SUPPORTED, list of handles
+       that are using this error handler (length: eh_refcount). */
+    mqs_taddr_t *eh_handles;
+
+    /* Address of the user-defined error handler (will be 0 for
+       predefined error handlers).  Note that each of the 3 C
+       callbacks contain an MPI handle; the debugger will need to
+       figure out the appropriate size for these types depending on
+       the platform and MPI implementation.  This value will be NULL
+       if MPIDBG_ERRH_INFO_PREDEFINED is set on the flags. */
+    mqs_taddr_t eh_callback_func;
+};
+
+/**************************************************************************
+ * Global variables
+ *
+ * mpidbg_dll_locations is in the MPI application; all others are in
+ * the DLL.
+ **************************************************************************/
+
+/* Array of filenames instantiated IN THE MPI APPLICATION (*NOT* in
+   the DLL) that provides an set of locations where DLLs may be found.
+   The last pointer in the array will be a NULL sentinel value.  The
+   debugger can scan the entries in the array, find one that matches
+   the debugger (by examining a) whether the dlopen works or not, and
+   b) if the dlopen succeeds, examine mpidbg_dll_is_big_endian and
+   mpidbg_dll_bitness), and try to dynamically open the dl_filename.
+   Notes:
+
+   1. It is not an error if a dl_filename either does not exist or is
+      otherwise un-openable (the debugger can just try the next
+      match).
+   2. This array values are not valid until MPIR_Breakpoint.
+   3. If a filename is absolute, the debugger will attempt to load
+      exactly that.  If the filename is relative, the debugger may try
+      a few prefix variations to find the DLL.
+ */
+extern char **mpidbg_dll_locations;
+
+/* Global variable *in the DLL* describing whether this DLL is big or
+   little endian (1 = big endian, 0 = little endian).  This value is
+   valid immediately upon opening of the DLL. */
+extern char mpidbg_dll_is_big_endian;
+
+/* Global variable *in the DLL* describing the bitness of the DLL (8,
+   16, 32, 64, ...).  This value is valid immediately upon opening of
+   the DLL. */
+extern char mpidbg_dll_bitness;
+
+/* Global variable *in the DLL* describing the DLL's capabilties with
+   regards to communicators.  This value is valid after a successfull
+   call to mpidbg_init_per_process(). */
+extern enum mpidbg_comm_capabilities_t mpidbg_comm_capabilities;
+
+/* Global variable *in the DLL* that is an array of MPI communicator
+   handle names -> handle mappings (the last entry in the array is
+   marked by a NULL string value).  For example, MPI_COMM_WORLD may
+   not appear as a symbol in an MPI process, but the debugger needs to
+   be able to map this name to a valid handle.  MPI implementations
+   not requiring this mapping can either have a NULL value for this
+   variable or have a single entry that has a NULL string value.  This
+   variable is not valid until after a successfull call to
+   mpidbg_init_per_process().  */
+extern struct mpidbg_name_map_t *mpidbg_comm_name_map;
+
+/* Global variable *in the DLL* describing the DLL's capabilties with
+   regards to error handlers.  This value is valid after a successfull
+   call to mpidbg_init_per_process(). */
+extern enum mpidbg_errhandler_capabilities_t mpidbg_errhandler_capabilities;
+
+/* Global variable *in the DLL* that is an array of MPI error handler
+   handle names -> handle mappings.  It is analogous to
+   mpidbg_comm_name_map; see above for details. */
+extern struct mpidbg_name_map_t *mpidbg_errhandler_name_map;
+
+/**************************************************************************
+ * Functions
+ **************************************************************************/
+
+/*-----------------------------------------------------------------------
+ * DLL infrastructure functions
+ *-----------------------------------------------------------------------*/
+
+/* This function must be called once before any other mpidbg_*()
+   function is called, and before any other global mpidbg_* data is
+   read.  It is only necessary to call this function once for a given
+   debugger instantiation.  This function will initialize all mpidbg
+   global state, to include setting all relevant global capability
+   flags.
+
+   Parameters:
+
+   IN: callbacks: Table of pointers to the debugger functions. The DLL
+                  need only save the pointer, the debugger promises to
+                  maintain the table of functions valid for as long as
+                  needed.  The table remains the property of the
+                  debugger, and should not be altered or deallocated
+                  by the DLL. This applies to all of the callback
+                  tables.
+
+   This function will return:
+
+   MPIDBG_SUCCESS: if all initialization went well
+   MPIDBG_ERR_*: if something went wrong.
+*/
+int mpidbg_init_once(const mqs_basic_callbacks *callbacks);
+
+/*-----------------------------------------------------------------------*/
+
+/* Query the DLL to find out what version of the interface it
+   supports. 
+
+   Parameters:
+
+   None.
+
+   This function will return:
+
+   MPIDBG_INTERFACE_VERSION
+*/
+
+int mpidbg_interface_version_compatibility(void);
+
+/*-----------------------------------------------------------------------*/
+
+/* Returns a string describing this DLL.
+
+   Parameters: 
+
+   None
+
+   This function will return:
+
+   A null-terminated string describing this DLL.
+*/   
+char *mpidbg_version_string(void);
+
+/*-----------------------------------------------------------------------*/
+
+/* Returns the address width that this DLL was compiled with.
+
+   Parameters: 
+
+   None
+
+   This function will return:
+
+   sizeof(mqs_taddr_t)
+*/
+
+int mpidbg_dll_taddr_width(void);
+
+/*-----------------------------------------------------------------------*/
+
+/* Setup debug information for a specific image, this must save the
+   callbacks (probably in the mqs_image_info), and use those functions
+   for accessing this image.
+
+   The DLL should use the mqs_put_image_info and mqs_get_image_info
+   functions to associate whatever information it wants to keep with
+   the image.  (For instance all of the type offsets it needs could be
+   kept here).  the debugger will call mqs_destroy_image_info when it
+   no longer wants to keep information about the given executable.
+ 
+   This will be called once for each executable image in the parallel
+   program.
+
+   Parameters:
+
+   IN: image: the application image.
+   IN: callbacks: Table of pointers to the debugger image-specific
+                  functions. The DLL need only save the pointer, the
+                  debugger promises to maintain the table of functions
+                  valid for as long as needed.  The table remains the
+                  property of the debugger, and should not be altered
+                  or deallocated by the DLL. This applies to all of
+                  the callback tables.
+   IN/OUT: handle_types: a pointer to a pre-allocated struct
+           containing mqs_types for each of the MPI handle types.
+           Must be filled in with results from mqs_find_type for each
+           MPI handle type.
+
+   This function will return:
+
+   MPIDBG_SUCCESS: if all initialization went well
+   MPIDBG_ERR_NOT_SUPPORTED: if the image does not support the MPIDBG
+                   interface.  In this case, no other mpidbg functions
+                   will be invoked on this image (not even
+                   mpidbg_finalize_per_image()).
+   MPIDBG_ERR_*: if something went wrong.
+*/
+int mpidbg_init_per_image(mqs_image *image,
+                          const mqs_image_callbacks *callbacks,
+                          struct mpidbg_handle_info_t *handle_types);
+
+/* This function will be called once when an application image that
+   previously had mpidbg_init_per_image() successfully invoked that is
+   now ending (e.g., the debugger is exiting, the debugger has
+   unloaded this image, etc.).  This function can be used to clean up
+   any image-specific data.
+
+   Parameters:
+
+   IN: image: the application image.
+   IN: image_info: the info associated with the application image.
+*/
+void mpidbg_finalize_per_image(mqs_image *image, mqs_image_info *image_info);
+
+/*-----------------------------------------------------------------------*/
+
+/* This function will only be called if mpidbg_init_per_image()
+   returned successfully, indicating that the image contains
+   information for MPI handle information.  If you cannot tell whether
+   a process will have MPI handle information in it by examining the
+   image, you should return SUCCESS from mpidbg_init_per_image() and
+   use this function to check whether MPI handle information is
+   available in the process.
+
+   Set up whatever process specific information we need.  For instance
+   addresses of global variables should be handled here rather than in
+   the image information, because if data may be in dynamic libraries
+   which could end up mapped differently in different processes.
+
+   Note that certain global variables are not valid until after this
+   call completes successfully (see above; e.g.,
+   mpidbg_comm_capabilities, mpidbg_comm_name_mapping, etc.).
+
+   Parameters:
+
+   IN: process: the process
+   IN: callbacks: Table of pointers to the debugger process-specific
+                  functions. The DLL need only save the pointer, the
+                  debugger promises to maintain the table of functions
+                  valid for as long as needed.  The table remains the
+                  property of the debugger, and should not be altered
+                  or deallocated by the DLL. This applies to all of
+                  the callback tables.
+   IN/OUT: handle_types: the same handle_types that was passed to
+           mqs_init_per_image.  It can be left unaltered if the
+           results from mqs_init_per_image were sufficient, or
+           modified if necessary to be specific to this process.
+
+   This function will return:
+
+   MPIDBG_SUCCESS: if all initialization went well
+   MPIDBG_ERR_NOT_SUPPORTED: if the process does not support the MPIDBG
+                   interface.  In this case, no other mpidbg functions
+                   will be invoked on this image (not even
+                   mpidbg_finalize_per_process()).
+   MPIDBG_ERR_*: if something went wrong.
+*/
+int mpidbg_init_per_process(mqs_process *process, 
+                            const mqs_process_callbacks *callbacks,
+                            struct mpidbg_handle_info_t *handle_types);
+
+/* This function will be called once when an application image that
+   previously had mpidbg_init_per_process() successfully invoked that
+   is now ending (e.g., the debugger is exiting, the debugger has
+   stopped executing this process, etc.).  This function can be used
+   to clean up any process-specific data.
+
+   Parameters:
+
+   IN: process: the application process.
+   IN: process_info: the info associated with the application process.
+*/
+void mpidbg_finalize_per_process(mqs_process *process,
+                                 mqs_process_info *process_info);
+
+/*-----------------------------------------------------------------------
+ * MPI handle query functions
+ * MPI_Comm
+ *-----------------------------------------------------------------------*/
+
+/* Query a specific MPI_Comm handle and, if found and valid, allocate
+   a new instance of the mpidbg_comm_info_t struct and all of its
+   internal data, and fill it in with information about the underlying
+   corresponding MPI communicator object.
+
+   Parameters:
+
+   IN: image: image
+   IN: image_info: image info that was previously "put"
+   IN: process: process
+   IN: process_info: process info that was previously "put"
+   IN: comm: communicator handle
+   OUT: info: pointer to be filled with a newly-allocated struct
+              mpidbg_comm_info_t
+
+   This function will return:
+
+   MPIDBG_SUCCESS: if the handle is valid, was found, and the info
+                   parameter was filled in successfully.
+   MPIDBG_ERR_NOT_FOUND: if the handle is not valid / found.
+   MPIDBG_ERR_UNSUPPORTED: if this function is unsupported.
+*/
+int mpidbg_comm_query(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t c_comm, struct mpidbg_comm_info_t **info);
+
+/* Query function to turn a Fortran INTEGER handle into its equivalent
+   C handle (that can then be queried with mpidbg_comm_query()).
+   mqs_taddr_t is used in order to guarantee to be large enough to
+   hold a Fortran INTEGER.
+
+   Parameters:
+
+   IN: image: image
+   IN: image_info: image info that was previously "put"
+   IN: process: process
+   IN: process_info: process info that was previously "put"
+   IN: f77_comm: a zero-padded Fortran integer containing the Fortran
+                 handle of the communicator.
+   OUT: c_comm: a C handle suitable to pass to mpidbg_comm_query().
+
+   This function returns:
+
+   MPIDBG_SUCCESS: if the handle is valid, was found, and the c_comm
+                   parameter was filled in successfully.
+   MPIDBG_ERR_NOT_FOUND: if the handle is not valid / found.
+   MPIDBG_ERR_UNSUPPORTED: if this function is unsupported.
+*/
+int mpidbg_comm_f2c(mqs_image *image, mqs_image_info *image_info, 
+                    mqs_process *process, mqs_process_info *process_info,
+                    mqs_taddr_t f77_comm, mqs_taddr_t *c_comm);
+
+/* Query function to turn a C++ handle into its equivalent C handle
+   (that can then be queried with mpidbg_comm_query()).  Pass the
+   pointer to the object as the cxx_comm (because we can't pass the
+   object itself); we return the C handle.
+
+   --> JMS Need more discussion here -- George has some opinion.  He
+       thinks we don't need this.
+   Parameters:
+
+   IN: image: image
+   IN: image_info: image info that was previously "put"
+   IN: process: process
+   IN: process_info: process info that was previously "put"
+   IN: cxx_comm: a pointer to the MPI handle object
+   IN: comm_type: one of 0, MPIDBG_COMM_INFO_CARTESION,
+                  MPIDBG_COMM_INFO_GRAPH, or
+                  MPIDBG_COMM_INFO_INTERCOMM indicating whether the
+                  object is an MPI::Comm, MPI::Cartcomm,
+                  MPI::Graphcomm, or MPI::Intercomm.
+   OUT: c_comm: a C handle suitable to pass to mpidbg_comm_query().
+
+   This function returns:
+
+   MPIDBG_SUCCESS: if the handle is valid, was found, and the c_comm
+                   parameter was filled in successfully.
+   MPIDBG_ERR_NOT_FOUND: if the handle is not valid / found.
+   MPIDBG_ERR_UNSUPPORTED: if this function is unsupported.
+*/
+int mpidbg_comm_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t cxx_comm, 
+                      enum mpidbg_comm_info_bitmap_t comm_type,
+                      mqs_taddr_t *c_comm);
+
+/*-----------------------------------------------------------------------
+ * MPI handle query functions
+ * MPI_Errhandler
+ *-----------------------------------------------------------------------*/
+
+/* These functions are analogous to the mpidbg_comm_* functions, but
+   for MPI_Errhandler.  Note that there is no need for a
+   "errhandler_type" argument to the cxx2c function because
+   MPI::Errhandler has no derived classes. */
+
+int mpidbg_errhandler_query(mqs_image *image, mqs_image_info *image_info, 
+                            mqs_process *process, mqs_process_info *process_info,
+                            mqs_taddr_t errhandler,
+                            struct mpidbg_errhandler_info_t **info);
+int mpidbg_errhandler_f2c(mqs_image *image, mqs_image_info *image_info, 
+                          mqs_process *process, mqs_process_info *process_info,
+                          mqs_taddr_t f77_errhandler, 
+                          mqs_taddr_t *c_errhandler);
+int mpidbg_errhandler_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                            mqs_process *process, mqs_process_info *process_info,
+                            mqs_taddr_t cxx_errhandler, 
+                            mqs_taddr_t *c_errhandler);
+
+/*-----------------------------------------------------------------------
+ * MPI handle query functions
+ * MPI_Request
+ *-----------------------------------------------------------------------*/
+
+/* These functions are analogous to the mpidbg_comm_* functions, but
+   for MPI_Request. */
+
+int mpidbg_request_query(mqs_image *image, mqs_image_info *image_info, 
+                         mqs_process *process, mqs_process_info *process_info,
+                         mqs_taddr_t request,
+                         struct mpidbg_request_info_t **info);
+int mpidbg_request_f2c(mqs_image *image, mqs_image_info *image_info, 
+                       mqs_process *process, mqs_process_info *process_info,
+                       mqs_taddr_t f77_request, mqs_taddr_t *c_request);
+int mpidbg_request_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                         mqs_process *process, mqs_process_info *process_info,
+                         mqs_taddr_t cxx_request, 
+                         enum mpidbg_request_info_bitmap_t request_type,
+                         mqs_taddr_t *c_request);
+
+/*-----------------------------------------------------------------------
+ * MPI handle query functions
+ * MPI_Status
+ *-----------------------------------------------------------------------*/
+
+/* These functions are analogous to the mpidbg_comm_* functions, but
+   for MPI_Status. */
+
+int mpidbg_status_query(mqs_image *image, mqs_image_info *image_info, 
+                        mqs_process *process, mqs_process_info *process_info,
+                        mqs_taddr_t status,
+                        struct mpidbg_status_info_t **info);
+int mpidbg_status_f2c(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t f77_status, mqs_taddr_t *c_status);
+int mpidbg_status_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                        mqs_process *process, mqs_process_info *process_info,
+                        mqs_taddr_t cxx_status, 
+                        mqs_taddr_t *c_status);
+
+#endif /* __MPIDBG_INTERFACE_H__ */

ompi/debuggers/msgq_interface.h

@@ -1,3 +1,15 @@
+/*
+ * Copyright (c) 2007      Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
 /* $Header: /home/tv/src/mpi/src/mpi_interface.h,v 1.13 2003/03/12 14:03:42 jcownie Exp $ */
 /* $Locker:  $ */
 
@@ -108,6 +120,9 @@
 
 #include <stdio.h>				/* For FILENAME_MAX */
 
+/* No MPI2 support yet */
+#define FOR_MPI2 0
+
 #ifdef	__cplusplus
 extern "C" {
 #endif
@@ -256,6 +271,68 @@ enum mqs_status
   mqs_st_pending, mqs_st_matched, mqs_st_complete
 };
 
+/* Additional error codes and error string conversion. */
+enum {
+    err_silent_failure  = mqs_first_user_code,
+
+    err_no_current_communicator,
+    err_bad_request,
+    err_no_store,
+
+    err_failed_qhdr,
+    err_unexpected,
+    err_posted,
+
+    err_failed_queue,
+    err_first,
+
+    err_context_id,
+    err_tag,
+    err_tagmask,
+    err_lsrc,
+    err_srcmask,
+    err_next,
+    err_ptr,
+
+    err_missing_type,
+    err_missing_symbol,
+
+    err_db_shandle,
+    err_db_comm,
+    err_db_target,
+    err_db_tag,
+    err_db_data,
+    err_db_byte_length,
+    err_db_next,
+
+    err_failed_rhandle,
+    err_is_complete,
+    err_buf,
+    err_len,
+    err_s,
+
+    err_failed_status,
+    err_count,
+    err_MPI_SOURCE,
+    err_MPI_TAG,
+
+    err_failed_commlist,
+    err_sequence_number,
+    err_comm_first,
+
+    err_failed_communicator,
+    err_lrank_to_grank,
+    err_send_context,
+    err_recv_context,
+    err_comm_next,
+    err_comm_name,
+
+    err_all_communicators,
+    err_mpid_sends,
+    err_mpid_recvs,
+    err_group_corrupt
+};
+
 /* A structure to represent a communicator */
 typedef struct
 {

ompi/debuggers/ompi_common_dll.c

@@ -0,0 +1,442 @@
+/*
+ * Copyright (c) 2007-2008 Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
+/**********************************************************************
+ * Copyright (C) 2000-2004 by Etnus, LLC.
+ * Copyright (C) 1999 by Etnus, Inc.
+ * Copyright (C) 1997-1998 Dolphin Interconnect Solutions Inc.
+ *
+ * Permission is hereby granted to use, reproduce, prepare derivative
+ * works, and to redistribute to others.
+ *
+ *				  DISCLAIMER
+ *
+ * Neither Dolphin Interconnect Solutions, Etnus LLC, nor any of their
+ * employees, makes any warranty express or implied, or assumes any
+ * legal liability or responsibility for the accuracy, completeness,
+ * or usefulness of any information, apparatus, product, or process
+ * disclosed, or represents that its use would not infringe privately
+ * owned rights.
+ *
+ * This code was written by
+ * James Cownie: Dolphin Interconnect Solutions. <jcownie@dolphinics.com>
+ *               Etnus LLC <jcownie@etnus.com>
+ **********************************************************************/
+
+#include "ompi_config.h"
+
+#include "ompi_common_dll_defs.h"
+
+/* Basic callbacks into the debugger */
+const mqs_basic_callbacks *mqs_basic_entrypoints;
+
+#if defined(WORDS_BIGENDIAN)
+static int host_is_big_endian = 1;
+#else
+static int host_is_big_endian = 0;
+#endif
+
+/*
+ * Open MPI use a bunch of lists in order to keep track of the
+ * internal objects. We have to make sure we're able to find all of
+ * them in the image and compute their ofset in order to be able to
+ * parse them later.  We need to find the opal_list_item_t, the
+ * opal_list_t, the ompi_free_list_item_t, and the ompi_free_list_t.
+ *
+ * Once we have these offsets, we should make sure that we have access
+ * to all requests lists and types. We're looking here only at the
+ * basic type for the requests as they hold all the information we
+ * need to export to the debugger.
+ */
+int ompi_fill_in_type_info(mqs_image *image, char **message)
+{
+    char* missing_in_action;
+    mpi_image_info * i_info = (mpi_image_info *)mqs_get_image_info (image);
+
+    {
+        mqs_type* qh_type = mqs_find_type( image, "opal_list_item_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "opal_list_item_t";
+            goto type_missing;
+        }
+        i_info->opal_list_item_t.type = qh_type;
+        i_info->opal_list_item_t.size = mqs_sizeof(qh_type);
+        i_info->opal_list_item_t.offset.opal_list_next = mqs_field_offset(qh_type, "opal_list_next");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "opal_list_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "opal_list_t";
+            goto type_missing;
+        }
+        i_info->opal_list_t.type = qh_type;
+        i_info->opal_list_t.size = mqs_sizeof(qh_type);
+        i_info->opal_list_t.offset.opal_list_sentinel = mqs_field_offset(qh_type, "opal_list_sentinel");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_free_list_item_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_free_list_item_t";
+            goto type_missing;
+        }
+        /* This is just an overloaded opal_list_item_t */
+        i_info->ompi_free_list_item_t.type = qh_type;
+        i_info->ompi_free_list_item_t.size = mqs_sizeof(qh_type);
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_free_list_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_free_list_t";
+            goto type_missing;
+        }
+        i_info->ompi_free_list_t.type = qh_type;
+        i_info->ompi_free_list_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_free_list_t.offset.fl_elem_class = mqs_field_offset(qh_type, "fl_elem_class");
+        i_info->ompi_free_list_t.offset.fl_mpool = mqs_field_offset(qh_type, "fl_mpool");
+        i_info->ompi_free_list_t.offset.fl_elem_size = mqs_field_offset(qh_type, "fl_elem_size");
+        i_info->ompi_free_list_t.offset.fl_alignment = mqs_field_offset(qh_type, "fl_alignment");
+        i_info->ompi_free_list_t.offset.fl_allocations = mqs_field_offset(qh_type, "fl_allocations");
+        i_info->ompi_free_list_t.offset.fl_max_to_alloc = mqs_field_offset(qh_type, "fl_max_to_alloc");
+        i_info->ompi_free_list_t.offset.fl_num_per_alloc = mqs_field_offset(qh_type, "fl_num_per_alloc");
+        i_info->ompi_free_list_t.offset.fl_num_allocated = mqs_field_offset(qh_type, "fl_num_allocated");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "opal_hash_table_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "opal_hash_table_t";
+            goto type_missing;
+        }
+        i_info->opal_hash_table_t.type = qh_type;
+        i_info->opal_hash_table_t.size = mqs_sizeof(qh_type);
+        i_info->opal_hash_table_t.offset.ht_table = mqs_field_offset(qh_type, "ht_table");
+        i_info->opal_hash_table_t.offset.ht_table_size = mqs_field_offset(qh_type, "ht_table_size");
+        i_info->opal_hash_table_t.offset.ht_size = mqs_field_offset(qh_type, "ht_size");
+        i_info->opal_hash_table_t.offset.ht_mask = mqs_field_offset(qh_type, "ht_mask");
+    }
+    /*
+     * Now let's look for all types required for reading the requests.
+     */
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_request_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_request_t";
+            goto type_missing;
+        }
+        i_info->ompi_request_t.type = qh_type;
+        i_info->ompi_request_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_request_t.offset.req_type = mqs_field_offset(qh_type, "req_type");
+        i_info->ompi_request_t.offset.req_status = mqs_field_offset(qh_type, "req_status");
+        i_info->ompi_request_t.offset.req_complete = mqs_field_offset(qh_type, "req_complete");
+        i_info->ompi_request_t.offset.req_state = mqs_field_offset(qh_type, "req_state");
+        i_info->ompi_request_t.offset.req_f_to_c_index = mqs_field_offset(qh_type, "req_f_to_c_index");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_base_request_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_base_request_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_base_request_t.type = qh_type;
+        i_info->mca_pml_base_request_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_base_request_t.offset.req_addr = mqs_field_offset(qh_type, "req_addr");
+        i_info->mca_pml_base_request_t.offset.req_count = mqs_field_offset(qh_type, "req_count");
+        i_info->mca_pml_base_request_t.offset.req_peer = mqs_field_offset(qh_type, "req_peer");
+        i_info->mca_pml_base_request_t.offset.req_tag = mqs_field_offset(qh_type, "req_tag");
+        i_info->mca_pml_base_request_t.offset.req_comm = mqs_field_offset(qh_type, "req_comm");
+        i_info->mca_pml_base_request_t.offset.req_datatype = mqs_field_offset(qh_type, "req_datatype");
+        i_info->mca_pml_base_request_t.offset.req_proc = mqs_field_offset(qh_type, "req_proc");
+        i_info->mca_pml_base_request_t.offset.req_sequence = mqs_field_offset(qh_type, "req_sequence");
+        i_info->mca_pml_base_request_t.offset.req_type = mqs_field_offset(qh_type, "req_type");
+        i_info->mca_pml_base_request_t.offset.req_pml_complete = mqs_field_offset(qh_type, "req_pml_complete");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_base_send_request_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_base_send_request_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_base_send_request_t.type = qh_type;
+        i_info->mca_pml_base_send_request_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_base_send_request_t.offset.req_addr = mqs_field_offset(qh_type, "req_addr");
+        i_info->mca_pml_base_send_request_t.offset.req_bytes_packed = mqs_field_offset(qh_type, "req_bytes_packed");
+        i_info->mca_pml_base_send_request_t.offset.req_send_mode = mqs_field_offset(qh_type, "req_send_mode");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_base_recv_request_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_base_recv_request_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_base_recv_request_t.type = qh_type;
+        i_info->mca_pml_base_recv_request_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_base_recv_request_t.offset.req_bytes_packed = mqs_field_offset(qh_type, "req_bytes_packed");
+    }
+    /*
+     * Gather information about the received fragments and theirs headers.
+     */
+#if 0  /* Disabled until I find a better way */
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_ob1_common_hdr_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_ob1_common_hdr_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_ob1_common_hdr_t.type = qh_type;
+        i_info->mca_pml_ob1_common_hdr_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_ob1_common_hdr_t.offset.hdr_type = mqs_field_offset(qh_type, "hdr_type");
+        i_info->mca_pml_ob1_common_hdr_t.offset.hdr_flags = mqs_field_offset(qh_type, "hdr_flags");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_ob1_match_hdr_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_ob1_match_hdr_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_ob1_match_hdr_t.type = qh_type;
+        i_info->mca_pml_ob1_match_hdr_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_ob1_match_hdr_t.offset.hdr_common = mqs_field_offset(qh_type, "hdr_common");
+        i_info->mca_pml_ob1_match_hdr_t.offset.hdr_ctx    = mqs_field_offset(qh_type, "hdr_ctx");
+        i_info->mca_pml_ob1_match_hdr_t.offset.hdr_src    = mqs_field_offset(qh_type, "hdr_src");
+        i_info->mca_pml_ob1_match_hdr_t.offset.hdr_tag    = mqs_field_offset(qh_type, "hdr_tag");
+        i_info->mca_pml_ob1_match_hdr_t.offset.hdr_seq    = mqs_field_offset(qh_type, "hdr_seq");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_pml_ob1_recv_frag_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_pml_ob1_recv_frag_t";
+            goto type_missing;
+        }
+        i_info->mca_pml_ob1_recv_frag_t.type = qh_type;
+        i_info->mca_pml_ob1_recv_frag_t.size = mqs_sizeof(qh_type);
+        i_info->mca_pml_ob1_recv_frag_t.offset.hdr = mqs_field_offset(qh_type, "hdr");
+        i_info->mca_pml_ob1_recv_frag_t.offset.request = mqs_field_offset(qh_type, "request");
+    }
+#endif
+    /*
+     * And now let's look at the communicator and group structures.
+     */
+    {
+        mqs_type* qh_type = mqs_find_type( image, "opal_pointer_array_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "opal_pointer_array_t";
+            goto type_missing;
+        }
+        i_info->opal_pointer_array_t.type = qh_type;
+        i_info->opal_pointer_array_t.size = mqs_sizeof(qh_type);
+        i_info->opal_pointer_array_t.offset.lowest_free = mqs_field_offset(qh_type, "lowest_free");
+        i_info->opal_pointer_array_t.offset.number_free = mqs_field_offset(qh_type, "number_free");
+        i_info->opal_pointer_array_t.offset.size = mqs_field_offset(qh_type, "size");
+        i_info->opal_pointer_array_t.offset.addr = mqs_field_offset(qh_type, "addr");
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_communicator_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_communicator_t";
+            goto type_missing;
+        }
+        i_info->ompi_communicator_t.type = qh_type;
+        i_info->ompi_communicator_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_communicator_t.offset.c_name = mqs_field_offset(qh_type, "c_name");
+        i_info->ompi_communicator_t.offset.c_contextid = mqs_field_offset(qh_type, "c_contextid");
+        i_info->ompi_communicator_t.offset.c_my_rank = mqs_field_offset(qh_type, "c_my_rank" );
+        i_info->ompi_communicator_t.offset.c_local_group = mqs_field_offset(qh_type, "c_local_group" );
+        i_info->ompi_communicator_t.offset.c_remote_group = mqs_field_offset(qh_type, "c_remote_group" );
+        i_info->ompi_communicator_t.offset.c_flags = mqs_field_offset(qh_type, "c_flags" );
+
+        i_info->ompi_communicator_t.offset.c_f_to_c_index = mqs_field_offset(qh_type, "c_f_to_c_index" );
+
+        i_info->ompi_communicator_t.offset.c_topo_comm = mqs_field_offset(qh_type, "c_topo_comm" );
+        i_info->ompi_communicator_t.offset.c_keyhash = mqs_field_offset(qh_type, "c_keyhash" );
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "mca_topo_base_comm_1_0_0_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "mca_topo_base_comm_1_0_0_t";
+            goto type_missing;
+        }
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.type = qh_type;
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_ndims_or_nnodes = 
+            mqs_field_offset(qh_type, "mtc_ndims_or_nnodes");
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_dims_or_index = 
+            mqs_field_offset(qh_type, "mtc_dims_or_index");
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_periods_or_edges = 
+            mqs_field_offset(qh_type, "mtc_periods_or_edges" );
+        i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_reorder = 
+            mqs_field_offset(qh_type, "mtc_reorder" );
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_group_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_group_t";
+            goto type_missing;
+        }
+        i_info->ompi_group_t.type = qh_type;
+        i_info->ompi_group_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_group_t.offset.grp_proc_count = mqs_field_offset(qh_type, "grp_proc_count");
+        i_info->ompi_group_t.offset.grp_my_rank = mqs_field_offset(qh_type, "grp_my_rank");
+        i_info->ompi_group_t.offset.grp_flags = mqs_field_offset(qh_type, "grp_flags" );
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_status_public_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_status_public_t";
+            goto type_missing;
+        }
+        i_info->ompi_status_public_t.type = qh_type;
+        i_info->ompi_status_public_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_status_public_t.offset.MPI_SOURCE = mqs_field_offset(qh_type, "MPI_SOURCE");
+        i_info->ompi_status_public_t.offset.MPI_TAG = mqs_field_offset(qh_type, "MPI_TAG");
+        i_info->ompi_status_public_t.offset.MPI_ERROR = mqs_field_offset(qh_type, "MPI_ERROR" );
+        i_info->ompi_status_public_t.offset._count = mqs_field_offset(qh_type, "_count" );
+        i_info->ompi_status_public_t.offset._cancelled = mqs_field_offset(qh_type, "_cancelled" );
+    }
+    {
+        mqs_type* qh_type = mqs_find_type( image, "ompi_datatype_t", mqs_lang_c );
+        if( !qh_type ) {
+            missing_in_action = "ompi_datatype_t";
+            goto type_missing;
+        }
+        i_info->ompi_datatype_t.type = qh_type;
+        i_info->ompi_datatype_t.size = mqs_sizeof(qh_type);
+        i_info->ompi_datatype_t.offset.size = mqs_field_offset(qh_type, "size");
+        i_info->ompi_datatype_t.offset.name = mqs_field_offset(qh_type, "name");
+    }
+
+    /* All the types are here. Let's succesfully return. */
+    return mqs_ok;
+
+ type_missing:
+    /*
+     * One of the required types is missing in the image. We are
+     * unable to extract the information we need from the pointers. We
+     * did our best but here we're at our limit. Give up!
+     */
+    *message = missing_in_action;
+    printf( "The following type is missing %s\n", missing_in_action );
+    return err_missing_type;
+}
+
+/***********************************************************************
+ * Functions to access the image memory. They are specialized based    *
+ * on the type we want to access and the debugged process architecture *
+ ***********************************************************************/
+mqs_taddr_t ompi_fetch_pointer (mqs_process *proc, mqs_taddr_t addr, 
+                                mpi_process_info *p_info)
+{
+    int isize = p_info->sizes.pointer_size;
+    char buffer[8];                  /* ASSUME the type fits in 8 bytes */
+    mqs_taddr_t res = 0;
+
+    if (mqs_ok == mqs_fetch_data (proc, addr, isize, buffer))
+        mqs_target_to_host (proc, buffer, 
+                            ((char *)&res) + (host_is_big_endian ? sizeof(mqs_taddr_t)-isize : 0), 
+                            isize);
+
+    return res;
+} /* fetch_pointer */
+
+/***********************************************************************/
+mqs_tword_t ompi_fetch_int (mqs_process *proc, mqs_taddr_t addr, 
+                            mpi_process_info *p_info)
+{
+    int isize = p_info->sizes.int_size;
+    char buffer[8];                  /* ASSUME the type fits in 8 bytes */
+    mqs_tword_t res = 0;
+
+    if (mqs_ok == mqs_fetch_data (proc, addr, isize, buffer)) {
+        mqs_target_to_host (proc, buffer, 
+                            ((char *)&res) + (host_is_big_endian ? sizeof(mqs_tword_t)-isize : 0), 
+                            isize);
+    }
+    return res;
+} /* fetch_int */
+
+/***********************************************************************/
+mqs_tword_t ompi_fetch_bool(mqs_process *proc, mqs_taddr_t addr, 
+                            mpi_process_info *p_info)
+{
+    int isize = p_info->sizes.bool_size;
+    mqs_tword_t res = 0;
+
+    mqs_fetch_data (proc, addr, isize, &res);
+    return (0 == res ? 0 : 1);
+} /* fetch_bool */
+
+/***********************************************************************/
+mqs_taddr_t ompi_fetch_size_t(mqs_process *proc, mqs_taddr_t addr, 
+                              mpi_process_info *p_info)
+{
+    int isize = p_info->sizes.size_t_size;
+    char buffer[8];                  /* ASSUME the type fits in 8 bytes */
+    mqs_taddr_t res = 0;
+
+    if (mqs_ok == mqs_fetch_data (proc, addr, isize, buffer))
+        mqs_target_to_host (proc, buffer, 
+                            ((char *)&res) + (host_is_big_endian ? sizeof(mqs_taddr_t)-isize : 0), 
+                            isize);
+  
+    return res;
+} /* fetch_size_t */
+
+/***********************************************************************/
+
+int ompi_fetch_opal_pointer_array_info(mqs_process *proc, mqs_taddr_t addr, 
+                                       mpi_process_info *p_info,
+                                       int *size, int *lowest_free, 
+                                       int *number_free)
+{
+    mqs_image *image = mqs_get_image(proc);
+    mpi_image_info *i_info = (mpi_image_info *) mqs_get_image_info(image);
+
+    *size = ompi_fetch_int(proc, 
+                           addr + i_info->opal_pointer_array_t.offset.size,
+                           p_info);
+    *lowest_free = ompi_fetch_int(proc, 
+                                  addr + i_info->opal_pointer_array_t.offset.lowest_free,
+                                  p_info);
+    *number_free = ompi_fetch_int(proc, 
+                                  addr + i_info->opal_pointer_array_t.offset.number_free,
+                                  p_info);
+    return mqs_ok;
+}
+
+/***********************************************************************/
+
+int ompi_fetch_opal_pointer_array_item(mqs_process *proc, mqs_taddr_t addr, 
+                                       mpi_process_info *p_info, int index,
+                                       mqs_taddr_t *item)
+{
+    mqs_image *image = mqs_get_image(proc);
+    mpi_image_info *i_info = (mpi_image_info *) mqs_get_image_info(image);
+    int size, lowest_free, number_free;
+    mqs_taddr_t base;
+
+    if (index < 0) {
+        return mqs_no_information;
+    }
+
+    ompi_fetch_opal_pointer_array_info(proc, addr, p_info, &size, 
+                                       &lowest_free, &number_free);
+    if (index >= size) {
+        return mqs_no_information;
+    }
+
+    base = ompi_fetch_pointer(proc, 
+                              addr + i_info->opal_pointer_array_t.offset.addr,
+                              p_info);
+    *item = ompi_fetch_pointer(proc,
+                               base + index * p_info->sizes.pointer_size,
+                               p_info);
+
+    return mqs_ok;
+}

ompi/debuggers/ompi_common_dll_defs.h

@@ -0,0 +1,310 @@
+/*
+ * Copyright (c) 2007-2008 Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
+/**********************************************************************
+ * Copyright (C) 2000-2004 by Etnus, LLC.
+ * Copyright (C) 1999 by Etnus, Inc.
+ * Copyright (C) 1997-1998 Dolphin Interconnect Solutions Inc.
+ *
+ * Permission is hereby granted to use, reproduce, prepare derivative
+ * works, and to redistribute to others.
+ *
+ *				  DISCLAIMER
+ *
+ * Neither Dolphin Interconnect Solutions, Etnus LLC, nor any of their
+ * employees, makes any warranty express or implied, or assumes any
+ * legal liability or responsibility for the accuracy, completeness,
+ * or usefulness of any information, apparatus, product, or process
+ * disclosed, or represents that its use would not infringe privately
+ * owned rights.
+ *
+ * This code was written by
+ * James Cownie: Dolphin Interconnect Solutions. <jcownie@dolphinics.com>
+ *               Etnus LLC <jcownie@etnus.com>
+ **********************************************************************/
+
+#ifndef OMPI_COMMON_DLL_DEFS_H
+#define OMPI_COMMON_DLL_DEFS_H
+
+#include "msgq_interface.h"
+
+/***********************************************************************
+ * Information associated with a specific executable image.  Common
+ * across all DLLs.
+ */
+typedef struct 
+{
+    /* Functions needed here */
+    const struct mqs_image_callbacks * image_callbacks;
+
+    /* basic structures */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int opal_list_next;
+        } offset;
+    } opal_list_item_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int opal_list_sentinel;
+        } offset;
+    } opal_list_t;
+    struct {
+        mqs_type *type;
+        int size;
+    } ompi_free_list_item_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int fl_elem_class;    /* opal_class_t* */
+            int fl_mpool;         /* struct mca_mpool_base_module_t* */
+            int fl_elem_size;     /* size_t */
+            int fl_alignment;     /* size_t */
+            int fl_allocations;   /* opal_list_t */
+            int fl_max_to_alloc;  /* size_t */
+            int fl_num_per_alloc; /* size_t */
+            int fl_num_allocated; /* size_t */
+        } offset;
+    } ompi_free_list_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int ht_table;
+            int ht_table_size;
+            int ht_size;
+            int ht_mask;
+        } offset;
+    } opal_hash_table_t;
+    /* requests structures */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int req_type;
+            int req_status;
+            int req_complete;
+            int req_state;
+            int req_f_to_c_index;
+        } offset;
+    } ompi_request_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int req_addr;
+            int req_count;
+            int req_peer;
+            int req_tag;
+            int req_comm;
+            int req_datatype;
+            int req_proc;
+            int req_sequence;
+            int req_type;
+            int req_pml_complete;
+        } offset;
+    } mca_pml_base_request_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int req_addr;
+            int req_bytes_packed;
+            int req_send_mode;
+        } offset;
+    } mca_pml_base_send_request_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int req_bytes_packed;
+        } offset;
+    } mca_pml_base_recv_request_t;
+#if 0
+    /* fragments for unexpected messages (as well as theirs headers) */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int hdr;
+            int request;
+        } offset;
+    } mca_pml_ob1_recv_frag_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int hdr_type;
+            int hdr_flags;
+        } offset;
+    } mca_pml_ob1_common_hdr_t;
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int hdr_common;
+            int hdr_ctx;
+            int hdr_src;
+            int hdr_tag;
+            int hdr_seq;
+        } offset;
+    } mca_pml_ob1_match_hdr_t;
+#endif
+    /* opal_pointer_array structure */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int lowest_free;
+            int number_free;
+            int size;
+            int addr;
+        } offset;
+    } opal_pointer_array_t;
+    /* group structure */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int grp_proc_count;
+            int grp_my_rank;
+            int grp_flags;
+        } offset;
+    } ompi_group_t;
+    /* communicator structure */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int c_name;
+            int c_contextid;
+            int c_my_rank;
+            int c_local_group;
+            int c_remote_group;
+            int c_flags;
+            int c_f_to_c_index;
+            int c_topo_comm;
+            int c_keyhash;
+        } offset;
+    } ompi_communicator_t;
+    /* base topology information in a communicator */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int mtc_ndims_or_nnodes;
+            int mtc_dims_or_index;
+            int mtc_periods_or_edges;
+            int mtc_reorder;
+        } offset;
+    } ompi_mca_topo_base_comm_1_0_0_t;
+    /* MPI_Status */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int MPI_SOURCE;
+            int MPI_TAG;
+            int MPI_ERROR;
+            int _count;
+            int _cancelled;
+        } offset;
+    } ompi_status_public_t;
+    /* datatype structure */
+    struct {
+        mqs_type *type;
+        int size;
+        struct {
+            int size;
+            int name;
+        } offset;
+    } ompi_datatype_t;
+
+    /* For the caller to hang their own stuff */
+    void *extra;
+} mpi_image_info; 
+
+/***********************************************************************/
+/* Information for a single process.  Common across all DLLs.
+ */
+typedef struct 
+{
+    const struct mqs_process_callbacks * process_callbacks; /* Functions needed here */
+
+    mqs_target_type_sizes sizes;			/* Process architecture information */
+
+    /* For the caller to hang their own stuff */
+    void *extra;
+} mpi_process_info;
+
+/**********************************************************************/
+/* Macros to make it transparent that we're calling the TV functions
+ * through function pointers.
+ */
+#define mqs_malloc           (mqs_basic_entrypoints->mqs_malloc_fp)
+#define mqs_free             (mqs_basic_entrypoints->mqs_free_fp)
+#define mqs_prints           (mqs_basic_entrypoints->mqs_dprints_fp)
+#define mqs_put_image_info   (mqs_basic_entrypoints->mqs_put_image_info_fp)
+#define mqs_get_image_info   (mqs_basic_entrypoints->mqs_get_image_info_fp)
+#define mqs_put_process_info (mqs_basic_entrypoints->mqs_put_process_info_fp)
+#define mqs_get_process_info (mqs_basic_entrypoints->mqs_get_process_info_fp)
+
+/* These macros *RELY* on the function already having set up the conventional
+ * local variables i_info or p_info.
+ */
+#define mqs_find_type        (i_info->image_callbacks->mqs_find_type_fp)
+#define mqs_field_offset     (i_info->image_callbacks->mqs_field_offset_fp)
+#define mqs_sizeof           (i_info->image_callbacks->mqs_sizeof_fp)
+#define mqs_get_type_sizes   (i_info->image_callbacks->mqs_get_type_sizes_fp)
+#define mqs_find_function    (i_info->image_callbacks->mqs_find_function_fp)
+#define mqs_find_symbol      (i_info->image_callbacks->mqs_find_symbol_fp)
+
+#define mqs_get_image        (p_info->process_callbacks->mqs_get_image_fp)
+#define mqs_get_global_rank  (p_info->process_callbacks->mqs_get_global_rank_fp)
+#define mqs_fetch_data       (p_info->process_callbacks->mqs_fetch_data_fp)
+#define mqs_target_to_host   (p_info->process_callbacks->mqs_target_to_host_fp)
+
+/* Basic callbacks into the debugger */
+extern const mqs_basic_callbacks *mqs_basic_entrypoints;
+
+/* OMPI-specific functions */
+int ompi_fill_in_type_info(mqs_image *image, char **message);
+
+/* Fetch a pointer from the process */
+mqs_taddr_t ompi_fetch_pointer(mqs_process *proc, mqs_taddr_t addr, 
+                               mpi_process_info *p_info);
+
+/* Fetch an int from the process */
+mqs_tword_t ompi_fetch_int(mqs_process *proc, mqs_taddr_t addr, 
+                           mpi_process_info *p_info);
+
+/* Fetch a bool from the process */
+mqs_tword_t ompi_fetch_bool(mqs_process *proc, mqs_taddr_t addr, 
+                            mpi_process_info *p_info);
+
+/* Fetch a size_t from the process */
+mqs_taddr_t ompi_fetch_size_t(mqs_process *proc, mqs_taddr_t addr, 
+                              mpi_process_info *p_info);
+
+/* Helpers to fetch stuff from an opal_pointer_array_t */
+int ompi_fetch_opal_pointer_array_info(mqs_process *proc, mqs_taddr_t addr, 
+                                       mpi_process_info *p_info,
+                                       int *size, int *lowest_free, 
+                                       int *number_free);
+int ompi_fetch_opal_pointer_array_item(mqs_process *proc, mqs_taddr_t addr, 
+                                       mpi_process_info *p_info, int index,
+                                       mqs_taddr_t *item);
+#endif

ompi/debuggers/ompi_debuggers.c

@@ -10,6 +10,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2007      Cisco, Inc.  All rights resereved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -18,7 +19,8 @@
  */
 
 /**
- * MPI portion of debugger support: TotalView
+ * MPI portion of debugger support: initially based on the
+ * TotalView/Etnus API for debuggers to attach to MPI jobs.
  */
 
 #include "ompi_config.h"
@@ -26,8 +28,22 @@
 #ifdef HAVE_UNISTD_H
 #include <unistd.h>
 #endif  /* HAVE_UNISTD_H */
+#ifdef HAVE_DIRENT_H
+#include <dirent.h>
+#endif
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#ifdef HAVE_SYS_STAT_H
+#include <sys/stat.h>
+#endif
+#ifdef HAVE_UNISTD_H
+#include <unistd.h>
+#endif
 
 #include "opal/mca/base/base.h"
+#include "opal/util/argv.h"
+#include "opal/mca/installdirs/installdirs.h"
 #include "debuggers.h"
 /**
  * A lot of include files that are required by al optimized builds in order
@@ -51,9 +67,13 @@
 OMPI_DECLSPEC int MPIR_being_debugged = 0;
 OMPI_DECLSPEC volatile int MPIR_debug_gate = 0;
 OMPI_DECLSPEC volatile int MPIR_debug_state = 0;
-#if defined(OMPI_TV_DLL)
+#if defined(OMPI_MSGQ_DLL)
-OMPI_DECLSPEC char MPIR_dll_name[] = OMPI_TV_DLL;
+/* This variable is old/deprecated -- the mpimsgq_dll_locations[]
-#endif  /* defined(OMPI_TV_DLL) */
+   method is preferred because it's more flexible */
+OMPI_DECLSPEC char MPIR_dll_name[] = OMPI_MSGQ_DLL;
+#endif  /* defined(OMPI_MSGQ_DLL) */
+OMPI_DECLSPEC char **mpidbg_dll_locations = NULL;
+OMPI_DECLSPEC char **mpimsgq_dll_locations = NULL;
 
 OMPI_DECLSPEC int MPIR_debug_typedefs_sizeof[] = {
     sizeof(short),
@@ -95,21 +115,78 @@ OMPI_DECLSPEC ompi_group_t* ompi_group_t_type_inclusion = NULL;
 OMPI_DECLSPEC ompi_status_public_t* ompi_status_public_t_type_inclusion = NULL;
 OMPI_DECLSPEC ompi_datatype_t* ompi_datatype_t_type_inclusion = NULL;
 
-/**
+/* Check for a file in few dirrect ways for portability */
- * Wait for a TotalView-like debugger if asked.
+static void check(char *dir, char *file, char **locations) 
- */
-void ompi_wait_for_totalview(void)
 {
-    int wait_for_totalview;
+    char *str;
+
+    asprintf(&str, "%s/%s.so", dir, file);
+    
+#if defined(HAVE_SYS_STAT_H)
+    {
+        struct stat buf;
+        
+        /* Use stat() */
+        if (0 == stat(str, &buf)) {
+            opal_argv_append_nosize(&locations, file);
+        }
+    }
+#else
+    {
+        FILE *fp;
+        
+        /* Just try to open the file */
+        if (NULL != (fp = fopen(str, "r"))) {
+            fclose(fp);
+            opal_argv_append_nosize(&locations, file);
+        }
+    }
+#endif /* defined(HAVE_SYS_STAT_H) */
+
+    free(str);
+}
+
+
+/**
+ * Wait for a debugger if asked.
+ */
+void ompi_wait_for_debugger(void)
+{
+    int i, wait_for_debugger, wait_for_tv;
+    char *a, *b, **dirs;
 
     /* Do we need to wait for a TotalView-like debugger? */
     mca_base_param_reg_int_name("orte",
-                                "mpi_wait_for_totalview",
+                                "mpi_wait_for_debugger",
                                 "Whether the MPI application "
                                 "should wait for a debugger or not",
                                 false, false, (int) false,
-                                &wait_for_totalview);
+                                &wait_for_debugger);
-    if (wait_for_totalview) {
+    mca_base_param_reg_int_name("orte",
+                                "mpi_wait_for_totalview",
+                                "Deprecated synonym for mpi_wait_for_debugger",
+                                false, false, (int) false,
+                                &wait_for_tv);
+    wait_for_debugger |= wait_for_tv;
+
+    a = strdup(opal_install_dirs.pkglibdir);
+    mca_base_param_reg_string_name("ompi",
+                                   "debugger_dll_path",
+                                   "List of directories where MPI_INIT should search for debugger plugins",
+                                   false, false, a, &b);
+    free(a);
+
+    /* Search the directory for MPI debugger DLLs */
+    if (NULL != b) {
+        dirs = opal_argv_split(b, ':');
+        for (i = 0; dirs[i] != NULL; ++i) {
+            check(dirs[i], OMPI_MPIHANDLES_DLL_PREFIX, mpidbg_dll_locations);
+            check(dirs[i], OMPI_MSGQ_DLL_PREFIX, mpimsgq_dll_locations);
+        }
+    }
+
+    /* If we're waiting for the debugger, then, well, wait for it.  :-) */
+    if (wait_for_debugger) {
         while (MPIR_debug_gate == 0) {
 #if defined(__WINDOWS__)
             Sleep(100);     /* milliseconds */
@@ -119,3 +196,4 @@ void ompi_wait_for_totalview(void)
         }
     }
 }
+

ompi/debuggers/ompi_mpihandles_dll.c

@@ -0,0 +1,751 @@
+/*
+ * Copyright (c) 2007-2008 Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
+/**********************************************************************
+ * Copyright (C) 2000-2004 by Etnus, LLC.
+ * Copyright (C) 1999 by Etnus, Inc.
+ * Copyright (C) 1997-1998 Dolphin Interconnect Solutions Inc.
+ *
+ * Permission is hereby granted to use, reproduce, prepare derivative
+ * works, and to redistribute to others.
+ *
+ *				  DISCLAIMER
+ *
+ * Neither Dolphin Interconnect Solutions, Etnus LLC, nor any of their
+ * employees, makes any warranty express or implied, or assumes any
+ * legal liability or responsibility for the accuracy, completeness,
+ * or usefulness of any information, apparatus, product, or process
+ * disclosed, or represents that its use would not infringe privately
+ * owned rights.
+ *
+ * This code was written by
+ * James Cownie: Dolphin Interconnect Solutions. <jcownie@dolphinics.com>
+ *               Etnus LLC <jcownie@etnus.com>
+ **********************************************************************/
+
+#include "ompi_config.h"
+
+#if defined(HAVE_STRING_H)
+#include <string.h>
+#endif  /* defined(HAVE_STRING_H) */
+#if defined(HAVE_STDLIB_H)
+#include <stdlib.h>
+#endif  /* defined(HAVE_STDLIB_H) */
+
+#include "ompi/mca/pml/base/pml_base_request.h"
+#include "mpihandles_interface.h"
+#include "ompi_mpihandles_dll_defs.h"
+#include "ompi/communicator/communicator.h"
+#include "ompi/group/group.h"
+
+
+#define OPAL_ALIGN(x,a,t) (((x)+((t)(a)-1)) & ~(((t)(a)-1)))
+
+/* Globals that the debugger expects to find in the DLL */
+#if defined(WORDS_BIGENDIAN)
+char mpidbg_dll_is_big_endian = 1;
+#else
+char mpidbg_dll_is_big_endian = 0;
+#endif
+char mpidbg_dll_bitness = (char) (sizeof(void*) * 8);
+enum mpidbg_comm_capabilities_t mpidbg_comm_capabilities = 0;
+struct mpidbg_name_map_t *mpidbg_comm_name_map = NULL;
+enum mpidbg_errhandler_capabilities_t mpidbg_errhandler_capabilities = 0;
+struct mpidbg_name_map_t *mpidbg_errhandler_name_map = NULL;
+enum mpidbg_request_capabilities_t mpidbg_request_capabilities = 0;
+struct mpidbg_name_map_t *mpidbg_request_name_map = NULL;
+enum mpidbg_status_capabilities_t mpidbg_status_capabilities = 0;
+struct mpidbg_name_map_t *mpidbg_status_name_map = NULL;
+
+/* Temporary workaround for making Totalview to load these symbols in
+   the library when this is compiled with the Sun Studio C compiler */
+#if defined(__SUNPRO_C)
+bool opal_uses_threads;
+bool opal_mutex_check_locks;
+volatile int32_t opal_progress_thread_count;
+int opal_progress_spin_count;
+#endif
+
+/*---------------------------------------------------------------------*/
+
+/* Small helper function: allocate a map of a given length */
+static struct mpidbg_name_map_t *alloc_map(mqs_image *image, int len)
+{
+    mpi_image_info *i_info = (mpi_image_info *) mqs_get_image_info(image);
+    struct mpidbg_name_map_t *m = NULL;
+
+    if (NULL != i_info) {
+        m = mqs_malloc(len * sizeof(struct mpidbg_name_map_t));
+    }
+
+    return m;
+}
+
+/* Small helper function: look up a symbol, and if we find it, put it
+   in a map entry */
+static void fill_map(mqs_image *image, 
+                     char *public_name, char *private_name,
+                     struct mpidbg_name_map_t *map)
+{
+    mqs_taddr_t value;
+    mpi_image_info *i_info = (mpi_image_info *) mqs_get_image_info(image);
+
+    if (NULL != i_info) {
+        map->map_name = strdup(public_name);
+        if (NULL != private_name) {
+            if (mqs_ok == mqs_find_symbol(image, private_name, &value)) {
+                map->map_handle = value;
+                return;
+            }
+        } else {
+            map->map_handle = 0;
+            return;
+        }
+    }
+
+    printf("OMPI MPI handles DLL: fill_map: Unable to find symbol: %s\n", 
+           private_name);
+}
+
+/* Helper function to lookup MPI attributes and fill an
+   mpidbg_attribute_pair_t array with their keys/values */
+static int fill_attributes(int *num_attrs, 
+                           struct mpidbg_attribute_pair_t **attrs, 
+                           mqs_taddr_t table)
+{
+    /* JMS fill me in */
+    return mqs_ok;
+}
+
+/*---------------------------------------------------------------------*/
+
+int mpidbg_init_once(const mqs_basic_callbacks *cb)
+{
+    mqs_basic_entrypoints = cb;
+    printf("mpidbg_init_once\n");
+    return MPIDBG_SUCCESS;
+}
+
+/*---------------------------------------------------------------------*/
+
+/* Returns the fixed value */
+int mpidbg_interface_version_compatibility(void)
+{
+    printf("mpidbg_interface_version_compatibility\n");
+    return MPIDBG_INTERFACE_VERSION;
+}
+
+
+/* Returns a string specific to OMPI */
+char *mpidbg_version_string(void)
+{
+    printf("mpidbg_version_string\n");
+    return "Open MPI handle interpretation support for parallel"
+           " debuggers compiled on " __DATE__;
+}
+
+
+/* So the debugger can tell what interface width the library was
+   compiled with */
+int mpidbg_dll_taddr_width(void)
+{
+    printf("mpidbg_dll_taddr_width\n");
+    return sizeof(mqs_taddr_t);
+}
+
+/*---------------------------------------------------------------------*/
+
+/* Once-per-image setup */
+int mpidbg_init_per_image(mqs_image *image, const mqs_image_callbacks *icb,
+                          struct mpidbg_handle_info_t *handle_types)
+{
+    char **message;
+    mpi_image_info *i_info = 
+        (mpi_image_info *) mqs_malloc(sizeof(mpi_image_info));
+    printf("mpidbg_init_per_image\n");
+
+    if (NULL == i_info) {
+        printf("mpidbg_init_per_image: malloc failed!\n");
+        return MPIDBG_ERR_NO_MEM;
+    }
+
+    memset((void *)i_info, 0, sizeof(mpi_image_info));
+    /* Before we do *ANYTHING* else */
+    i_info->image_callbacks = icb;
+
+    /* Nothing extra (yet) */
+    i_info->extra = NULL;
+
+    /* Save the info */
+    mqs_put_image_info(image, (mqs_image_info *)i_info);
+
+    /* Fill in the OMPI type information */
+    if (mqs_ok != ompi_fill_in_type_info(image, message)) {
+        printf("mpidbg_init_per_image: failed to get all type info\n");
+        return MPIDBG_ERR_NOT_SUPPORTED;
+    }
+
+    /* Fill in the handle_types struct with our types */
+    /* JMS: "MPI_Aint" is a typedef -- is that enough?  (the actual
+       type is a #define, so it's not easy to put into the
+       mqs_find_type call as a string) */
+    handle_types->hi_c_aint = mqs_find_type(image, "MPI_Aint", mqs_lang_c);
+    /* JMS: these ompi types are just the "foo" types; but OMPI MPI
+       types are all "foo*"'s -- is this right?  If this is wrong, I
+       *suspect* that something like the following may be right:
+       
+       handle_types->hi_c_comm = mqs_find_type(image, "ompi_communicator_t*", mqs_lang_c);
+
+       Need to confirm this with the DDT guys...
+    */
+    handle_types->hi_c_comm = i_info->ompi_communicator_t.type;
+    handle_types->hi_c_datatype = i_info->ompi_datatype_t.type;
+    handle_types->hi_c_errhandler = 
+        mqs_find_type(image, "ompi_errhandler_t", mqs_lang_c);
+    handle_types->hi_c_file = 
+        mqs_find_type(image, "ompi_file_t", mqs_lang_c);
+    handle_types->hi_c_group = i_info->ompi_group_t.type;
+    handle_types->hi_c_info = 
+        mqs_find_type(image, "ompi_info_t", mqs_lang_c);
+    /* JMS: "MPI_Offset" is a typedef (see comment about MPI_Aint above) */
+    handle_types->hi_c_offset =
+        mqs_find_type(image, "MPI_Offset", mqs_lang_c);
+    handle_types->hi_c_op = 
+        mqs_find_type(image, "ompi_op_t", mqs_lang_c);
+    handle_types->hi_c_request = i_info->ompi_request_t.type;
+    handle_types->hi_c_status = i_info->ompi_status_public_t.type;
+    handle_types->hi_c_win = 
+        mqs_find_type(image, "ompi_win_t", mqs_lang_c);
+
+    /* MPI::Aint is a typedef to MPI_Aint */
+    handle_types->hi_cxx_aint = handle_types->hi_cxx_aint;
+    handle_types->hi_cxx_comm =
+        mqs_find_type(image, "MPI::Comm", mqs_lang_cplus);
+    handle_types->hi_cxx_intracomm =
+        mqs_find_type(image, "MPI::Intracomm", mqs_lang_cplus);
+    handle_types->hi_cxx_intercomm =
+        mqs_find_type(image, "MPI::Intercomm", mqs_lang_cplus);
+    handle_types->hi_cxx_graphcomm =
+        mqs_find_type(image, "MPI::Graphcomm", mqs_lang_cplus);
+    handle_types->hi_cxx_cartcomm =
+        mqs_find_type(image, "MPI::Cartcomm", mqs_lang_cplus);
+    handle_types->hi_cxx_datatype =
+        mqs_find_type(image, "MPI::Datatype", mqs_lang_cplus);
+    handle_types->hi_cxx_errhandler =
+        mqs_find_type(image, "MPI::Errhandler", mqs_lang_cplus);
+    handle_types->hi_cxx_file =
+        mqs_find_type(image, "MPI::File", mqs_lang_cplus);
+    handle_types->hi_cxx_group =
+        mqs_find_type(image, "MPI::Group", mqs_lang_cplus);
+    handle_types->hi_cxx_info =
+        mqs_find_type(image, "MPI::Info", mqs_lang_cplus);
+    /* MPI::Offset is a typedef to MPI_Offset */
+    handle_types->hi_cxx_offset = handle_types->hi_c_offset;
+    handle_types->hi_cxx_op =
+        mqs_find_type(image, "MPI::Op", mqs_lang_cplus);
+    handle_types->hi_cxx_request =
+        mqs_find_type(image, "MPI::Request", mqs_lang_cplus);
+    handle_types->hi_cxx_prequest =
+        mqs_find_type(image, "MPI::Prequest", mqs_lang_cplus);
+    handle_types->hi_cxx_grequest =
+        mqs_find_type(image, "MPI::Grequest", mqs_lang_cplus);
+    handle_types->hi_cxx_status =
+        mqs_find_type(image, "MPI::Status", mqs_lang_cplus);
+    handle_types->hi_cxx_win =
+        mqs_find_type(image, "MPI::Win", mqs_lang_cplus);
+
+    /* Tell the debuger what capabilities we have */
+    mpidbg_comm_capabilities = 
+        MPIDBG_COMM_CAP_BASIC | 
+        MPIDBG_COMM_CAP_STRING_NAMES | 
+        MPIDBG_COMM_CAP_FREED_HANDLE | 
+        MPIDBG_COMM_CAP_FREED_OBJECT;
+    mpidbg_errhandler_capabilities =
+        MPIDBG_ERRH_CAP_BASIC | 
+        MPIDBG_ERRH_CAP_STRING_NAMES | 
+        MPIDBG_ERRH_CAP_FREED_HANDLE | 
+        MPIDBG_ERRH_CAP_FREED_OBJECT;
+    mpidbg_request_capabilities = 
+        MPIDBG_REQUEST_CAP_BASIC;
+    mpidbg_status_capabilities = 
+        MPIDBG_STATUS_CAP_BASIC;
+
+    /* All done */
+    printf("mpidbg_init_per_image: init succeeded -- ready!\n");
+    return MPIDBG_SUCCESS;
+}
+
+
+/* This image is now dead; free all the state associated with it */
+void mpidbg_finalize_per_image(mqs_image *image, mqs_image_info *info)
+{
+    mpi_image_info *i_info = (mpi_image_info *)info;
+
+    printf("mpidbg_finalize_per_image\n");
+    if (NULL != i_info->extra) {
+        mqs_free(i_info->extra);
+    }
+    mqs_free(info);
+}
+
+/*---------------------------------------------------------------------*/
+
+/* Setup information needed for a specific process.  The debugger
+ * assumes that this will hang something onto the process, if nothing
+ * is attached to it, then TV will believe that this process has no
+ * message queue information.
+ */
+int mpidbg_init_per_process(mqs_process *process, 
+                            const mqs_process_callbacks *pcb,
+                            struct mpidbg_handle_info_t *handle_types)
+{ 
+    mqs_image *image;
+    mpi_image_info *i_info;
+
+    /* Extract the addresses of the global variables we need and save
+       them away */
+    mpi_process_info *p_info = 
+        (mpi_process_info *) mqs_malloc(sizeof(mpi_process_info));
+    printf("mpidbg_init_per_process\n");
+
+    if (NULL == p_info) {
+        return MPIDBG_ERR_NO_MEM;
+    }
+
+    /* Setup the callbacks first */
+    p_info->process_callbacks = pcb;
+
+    /* Nothing extra (yet) */
+    p_info->extra = NULL;
+
+    /* Now we can get the rest of the info */
+    image = mqs_get_image(process);
+    i_info = (mpi_image_info *) mqs_get_image_info(image);
+
+    /* Get process info sizes */
+    mqs_get_type_sizes (process, &p_info->sizes);
+
+    /* Save the info */
+    mqs_put_process_info(process, (mqs_process_info *) p_info);
+
+    /* Fill in pre-defined MPI handle name mappings (because OMPI uses
+       #define's for the pre-defined names, such as "#define
+       MPI_COMM_WORLD &ompi_mpi_comm_world"). */
+    /* Communicators */
+    mpidbg_comm_name_map = alloc_map(image, 4);
+    if (NULL != mpidbg_comm_name_map) {
+        int i = 0;
+        fill_map(image, "MPI_COMM_WORLD", "ompi_mpi_comm_world",
+                 &mpidbg_comm_name_map[i++]);
+        fill_map(image, "MPI_COMM_SELF", "ompi_mpi_comm_self",
+                 &mpidbg_comm_name_map[i++]);
+        fill_map(image, "MPI_COMM_NULL", "ompi_mpi_comm_null",
+                 &mpidbg_comm_name_map[i++]);
+
+        /* Sentinel value */
+        mpidbg_comm_name_map[i].map_name = NULL;
+    }
+
+    /* Error handlers */
+    mpidbg_errhandler_name_map = alloc_map(image, 4);
+    if (NULL != mpidbg_errhandler_name_map) {
+        int i = 0;
+        fill_map(image, "MPI_ERRORS_ARE_FATAL", "ompi_mpi_errors_are_fatal",
+                 &mpidbg_errhandler_name_map[i++]);
+        fill_map(image, "MPI_ERRORS_RETURN", "ompi_mpi_errors_return",
+                 &mpidbg_errhandler_name_map[i++]);
+        fill_map(image, "MPI_ERRHANDLER_NULL", "ompi_mpi_errhandler_null",
+                 &mpidbg_errhandler_name_map[i++]);
+        /* MPI::ERRORS_THROW_EXCEPTIONS exists as a symbol in OMPI; no
+           need to alias it here */
+
+        /* Sentinel value */
+        mpidbg_errhandler_name_map[i].map_name = NULL;
+    }
+
+    /* Requests */
+    mpidbg_request_name_map = alloc_map(image, 2);
+    if (NULL != mpidbg_request_name_map) {
+        int i = 0;
+        fill_map(image, "MPI_REQUEST_NULL", "ompi_request_null",
+                 &mpidbg_request_name_map[i++]);
+
+        /* Sentinel value */
+        mpidbg_request_name_map[i].map_name = NULL;
+    }
+
+    /* Statuses */
+    mpidbg_status_name_map = alloc_map(image, 2);
+    if (NULL != mpidbg_status_name_map) {
+        int i = 0;
+        fill_map(image, "MPI_STATUS_IGNORE", NULL,
+                 &mpidbg_status_name_map[i++]);
+
+        /* Sentinel value */
+        mpidbg_status_name_map[i].map_name = NULL;
+    }
+
+    /* All done */
+    return MPIDBG_SUCCESS;
+}
+
+
+/* This process is now done; free all the state associated with it */
+void mpidbg_finalize_per_process(mqs_process *process, mqs_process_info *info)
+{
+    mpi_process_info *p_info = (mpi_process_info *)info;
+
+    printf("mpidbg_finalize_per_process\n");
+    if (NULL != p_info->extra) {
+        mqs_free(p_info->extra);
+    }
+    mqs_free(info);
+}
+
+
+/*---------------------------------------------------------------------*/
+
+int mpidbg_comm_query(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t c_comm, struct mpidbg_comm_info_t **info)
+{
+    int flags;
+    mpi_image_info *i_info = (mpi_image_info*) image_info;
+    mpi_process_info *p_info = (mpi_process_info*) process_info;
+    mqs_taddr_t group, topo, keyhash;
+
+    /* Get the comm name */
+
+    *info = mqs_malloc(sizeof(struct mpidbg_comm_info_t));
+    if (NULL == *info) {
+        return MPIDBG_ERR_NO_MEM;
+    }
+    /* JMS temporarily zero everything out.  Remove this when we fill
+       in all the fields */
+    memset(*info, 0, sizeof(struct mpidbg_comm_info_t));
+    (*info)->comm_c_handle = c_comm;
+
+    printf("mpidbg_comm_query: %p\n", (void*) c_comm);
+    mqs_fetch_data(process, c_comm + i_info->ompi_communicator_t.offset.c_name,
+                   MPIDBG_MAX_OBJECT_NAME, (*info)->comm_name);
+
+    /* Get this process' rank in the comm */
+    (*info)->comm_rank = ompi_fetch_int(process, 
+                                        c_comm + i_info->ompi_communicator_t.offset.c_my_rank,
+                                        p_info);
+
+    /* Analyze the flags on the comm */
+    flags = ompi_fetch_int(process, 
+                           c_comm + i_info->ompi_communicator_t.offset.c_flags,
+                           p_info);
+    (*info)->comm_bitflags = 0;
+    if (MPI_PROC_NULL == (*info)->comm_rank) {
+        /* This communicator is MPI_COMM_NULL */
+        (*info)->comm_rank = (*info)->comm_size = 0;
+        (*info)->comm_bitflags |= MPIDBG_COMM_INFO_COMM_NULL;
+    } else if (0 != (flags & OMPI_COMM_INTER)) {
+        (*info)->comm_bitflags |= MPIDBG_COMM_INFO_INTERCOMM;
+    } else {
+        if (0 != (flags & OMPI_COMM_CART)) {
+            (*info)->comm_bitflags |= MPIDBG_COMM_INFO_CARTESIAN;
+        } else if (0 != (flags & OMPI_COMM_GRAPH)) {
+            (*info)->comm_bitflags |= MPIDBG_COMM_INFO_GRAPH;
+        }
+    }
+    if (0 != (flags & OMPI_COMM_ISFREED)) {
+        (*info)->comm_bitflags |= MPIDBG_COMM_INFO_FREED_HANDLE;
+    }
+    if (0 != (flags & OMPI_COMM_INTRINSIC)) {
+        (*info)->comm_bitflags |= MPIDBG_COMM_INFO_PREDEFINED;
+    }
+    if (0 != (flags & OMPI_COMM_INVALID)) {
+        (*info)->comm_bitflags |= MPIDBG_COMM_INFO_FREED_OBJECT;
+    }
+    
+    /* Look up the local group */
+    group = ompi_fetch_pointer(process, 
+                               c_comm + i_info->ompi_communicator_t.offset.c_local_group,
+                               p_info);
+    (*info)->comm_rank = ompi_fetch_int(process, 
+                                        group + i_info->ompi_group_t.offset.grp_my_rank,
+                                        p_info);
+    (*info)->comm_num_local_procs = ompi_fetch_int(process, 
+                                                   group + i_info->ompi_group_t.offset.grp_proc_count,
+                                                   p_info);
+
+    /* Fill in the comm_size with the size of the local group.  We'll
+       override below if this is an intercommunicator. */
+    (*info)->comm_size = (*info)->comm_num_local_procs;
+
+    /* JMS fill this in: waiting to decide between mpidbg_process_t
+       and mqs_process_location */
+    (*info)->comm_local_procs = NULL;
+
+    /* Look up the remote group (if relevant) */
+    if (0 != (flags & OMPI_COMM_INTER)) {
+        group = ompi_fetch_pointer(process, 
+                                   c_comm + i_info->ompi_communicator_t.offset.c_remote_group,
+                                   p_info);
+        (*info)->comm_num_remote_procs = ompi_fetch_int(process, 
+                                                        group + i_info->ompi_group_t.offset.grp_proc_count,
+                                                        p_info);
+        (*info)->comm_size = (*info)->comm_num_remote_procs;
+
+        /* JMS fill this in: waiting to decide between
+           mpidbg_process_t and mqs_process_location */
+        (*info)->comm_remote_procs = NULL;
+    } else {
+        (*info)->comm_num_remote_procs = 0;
+        (*info)->comm_remote_procs = NULL;
+    }
+
+    /* Fill in cartesian/graph info, if relevant.  The cartesian and
+       graph data is just slightly different from each other; it's
+       [slightly] easier (and less confusing!) to have separate
+       retrieval code blocks. */
+    topo = ompi_fetch_pointer(process, 
+                              c_comm + i_info->ompi_communicator_t.offset.c_topo_comm,
+                              p_info);
+    if (0 != topo && 
+        0 != ((*info)->comm_bitflags & MPIDBG_COMM_INFO_CARTESIAN)) {
+        int i, ndims, tmp;
+        mqs_taddr_t dims, periods;
+
+        /* Alloc space for copying arrays */
+        (*info)->comm_cart_num_dims = ndims =
+            ompi_fetch_int(process, 
+                           topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_ndims_or_nnodes,
+                           p_info);
+        (*info)->comm_cart_dims = mqs_malloc(ndims * sizeof(int));
+        if (NULL == (*info)->comm_cart_dims) {
+            return MPIDBG_ERR_NO_MEM;
+        }
+        (*info)->comm_cart_periods = mqs_malloc(ndims * sizeof(int8_t));
+        if (NULL == (*info)->comm_cart_periods) {
+            mqs_free((*info)->comm_cart_dims);
+            (*info)->comm_cart_dims = NULL;
+            return MPIDBG_ERR_NO_MEM;
+        }
+
+        /* Retrieve the dimension and periodic description data from
+           the two arrays on the image's communicator */
+        dims = ompi_fetch_pointer(process, 
+                                 topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_dims_or_index,
+                                 p_info);
+        periods = ompi_fetch_pointer(process, 
+                                 topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_periods_or_edges,
+                                 p_info);
+
+        for (i = 0; i < ndims; ++i) {
+            (*info)->comm_cart_dims[i] =
+                ompi_fetch_int(process, dims + (sizeof(int) * i), p_info);
+            tmp = ompi_fetch_int(process, periods + (sizeof(int) * i), p_info);
+            (*info)->comm_cart_periods[i] = (int8_t) tmp;
+            printf("mpidbg: cart comm: dimension %d: (length %d, periodic: %d)\n", i, (*info)->comm_cart_dims[i], tmp);
+        }
+    } else if (0 != topo &&
+               0 != ((*info)->comm_bitflags & MPIDBG_COMM_INFO_GRAPH)) {
+        int i, nnodes;
+        mqs_taddr_t index, edges;
+
+        /* Alloc space for copying the indexes */
+        (*info)->comm_graph_num_nodes = nnodes = 
+            ompi_fetch_int(process, 
+                           topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_ndims_or_nnodes,
+                           p_info);
+        (*info)->comm_graph_index = mqs_malloc(nnodes * sizeof(int));
+        if (NULL == (*info)->comm_graph_index) {
+            return MPIDBG_ERR_NO_MEM;
+        }
+
+        /* Retrieve the index data */
+        index = ompi_fetch_pointer(process, 
+                                 topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_dims_or_index,
+                                 p_info);
+        for (i = 0; i < nnodes; ++i) {
+            (*info)->comm_graph_index[i] = 
+                ompi_fetch_int(process, index + (sizeof(int) * i), p_info);
+        }
+
+        /* Allocate space for the edges */
+        (*info)->comm_graph_edges = mqs_malloc((*info)->comm_graph_index[(*info)->comm_graph_num_nodes - 1] * sizeof(int));
+        if (NULL == (*info)->comm_graph_edges) {
+            mqs_free((*info)->comm_graph_index);
+            (*info)->comm_graph_index = NULL;
+            return MPIDBG_ERR_NO_MEM;
+        }
+
+        /* Retrieve the edge data */
+        edges = ompi_fetch_pointer(process, 
+                                 topo + i_info->ompi_mca_topo_base_comm_1_0_0_t.offset.mtc_periods_or_edges,
+                                 p_info);
+        for (i = 0; 
+             i < (*info)->comm_graph_index[(*info)->comm_graph_num_nodes - 1]; 
+             ++i) {
+            (*info)->comm_graph_edges[i] = 
+                ompi_fetch_int(process, edges + (sizeof(int) * i), p_info);
+        }
+    }
+
+    /* Fortran handle */
+    (*info)->comm_fortran_handle = 
+        ompi_fetch_int(process, 
+                       c_comm + i_info->ompi_communicator_t.offset.c_f_to_c_index,
+                       p_info);
+    printf("mpdbg: comm fortran handle: %d\n", (*info)->comm_fortran_handle);
+
+    /* Fill in attributes */
+    keyhash = ompi_fetch_pointer(process, 
+                                 c_comm + i_info->ompi_communicator_t.offset.c_keyhash,
+                                 p_info);
+    fill_attributes(&((*info)->comm_num_attrs), &((*info)->comm_attrs),
+                    keyhash);
+
+    /* JMS temporary */
+    (*info)->comm_num_pending_requests = MPIDBG_ERR_NOT_SUPPORTED;
+    (*info)->comm_pending_requests = NULL;
+    (*info)->comm_num_derived_windows = MPIDBG_ERR_NOT_SUPPORTED;
+    (*info)->comm_derived_windows = NULL;
+    (*info)->comm_num_derived_files = MPIDBG_ERR_NOT_SUPPORTED;
+    (*info)->comm_derived_files = NULL;
+
+    return MPIDBG_SUCCESS;
+}
+
+int mpidbg_comm_f2c(mqs_image *image, mqs_image_info *image_info, 
+                    mqs_process *process, mqs_process_info *process_info,
+                    mqs_taddr_t f77_comm, mqs_taddr_t *c_comm)
+{
+    mqs_taddr_t comm_list;
+    mpi_image_info *i_info = (mpi_image_info *) image_info;
+    mpi_process_info *p_info = (mpi_process_info*) process_info;
+
+    mqs_find_symbol(image, "ompi_mpi_communicators", &comm_list);
+    if (mqs_ok != ompi_fetch_opal_pointer_array_item(process, comm_list, 
+                                                     p_info, f77_comm,
+                                                     c_comm) ||
+        NULL == c_comm) {
+        printf("mpidbg_comm_f2c: %lu -> not found\n", 
+               (long unsigned int) f77_comm);
+        return MPIDBG_ERR_NOT_FOUND;
+    }
+    printf("mpidbg_comm_f2c: %lu -> %lu\n", 
+           (long unsigned int) f77_comm, (long unsigned int) c_comm);
+    return MPIDBG_SUCCESS;
+}
+
+int mpidbg_comm_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t cxx_comm, 
+                      enum mpidbg_comm_info_bitmap_t comm_type,
+                      mqs_taddr_t *c_comm)
+{
+    /* David tells me that any type of communicator (MPI::Comm,
+       MPI::Intracomm, etc.) should have the offset to the mpi_comm
+       member in the same place. */
+    printf("mpidbg_comm_cxx2c: %p\n", (void*) cxx_comm);
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+/*---------------------------------------------------------------------*/
+
+int mpidbg_errhandler_query(mqs_image *image, mqs_image_info *image_info, 
+                            mqs_process *process, mqs_process_info *process_info,
+                            mqs_taddr_t c_errhandler, 
+                            struct mpidbg_errhandler_info_t **info)
+{
+    printf("mpidbg_errhandler_query: %p\n", (void*) c_errhandler);
+    printf("mpidbg_errhandler_query: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_errhandler_f2c(mqs_image *image, mqs_image_info *image_info, 
+                          mqs_process *process, mqs_process_info *process_info,
+                          mqs_taddr_t f77_errhandler, mqs_taddr_t *c_errhandler)
+{
+    printf("mpidbg_errhandler_f2c: %lu\n", (long unsigned int) f77_errhandler);
+    printf("mpidbg_errhandler_f2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_errhandler_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                            mqs_process *process, mqs_process_info *process_info,
+                            mqs_taddr_t cxx_errhandler, 
+                            mqs_taddr_t *c_errhandler)
+{
+    printf("mpidbg_errhandler_cxx2c: %p\n", (void*) cxx_errhandler);
+    printf("mpidbg_errhandler_cxx2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+/*---------------------------------------------------------------------*/
+
+int mpidbg_request_query(mqs_image *image, mqs_image_info *image_info, 
+                         mqs_process *process, mqs_process_info *process_info,
+                         mqs_taddr_t c_request, 
+                         struct mpidbg_request_info_t **info)
+{
+    printf("mpidbg_request_query: %p\n", (void*) c_request);
+    printf("mpidbg_request_query: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_request_f2c(mqs_image *image, mqs_image_info *image_info, 
+                       mqs_process *process, mqs_process_info *process_info,
+                       mqs_taddr_t f77_request, mqs_taddr_t *c_request)
+{
+    printf("mpidbg_request_f2c: %lu\n", (long unsigned int) f77_request);
+    printf("mpidbg_request_f2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_request_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                         mqs_process *process, mqs_process_info *process_info,
+                         mqs_taddr_t cxx_request, 
+                         enum mpidbg_request_info_bitmap_t request_type,
+                         mqs_taddr_t *c_request)
+{
+    printf("mpidbg_request_cxx2c: %p\n", (void*) cxx_request);
+    printf("mpidbg_request_cxx2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+/*---------------------------------------------------------------------*/
+
+int mpidbg_status_query(mqs_image *image, mqs_image_info *image_info, 
+                        mqs_process *process, mqs_process_info *process_info,
+                        mqs_taddr_t c_status, 
+                        struct mpidbg_status_info_t **info)
+{
+    printf("mpidbg_status_query: %p\n", (void*) c_status);
+    printf("mpidbg_status_query: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_status_f2c(mqs_image *image, mqs_image_info *image_info, 
+                      mqs_process *process, mqs_process_info *process_info,
+                      mqs_taddr_t f77_status, mqs_taddr_t *c_status)
+{
+    printf("mpidbg_status_f2c: %lu\n", (long unsigned int) f77_status);
+    printf("mpidbg_status_f2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}
+
+int mpidbg_status_cxx2c(mqs_image *image, mqs_image_info *image_info, 
+                        mqs_process *process, mqs_process_info *process_info,
+                        mqs_taddr_t cxx_status, 
+                        mqs_taddr_t *c_status)
+{
+    printf("mpidbg_status_cxx2c: %p\n", (void*) cxx_status);
+    printf("mpidbg_status_cxx2c: not [yet] found\n");
+    return MPIDBG_ERR_NOT_FOUND;
+}

ompi/debuggers/ompi_mpihandles_dll_defs.h

@@ -0,0 +1,42 @@
+/*
+ * Copyright (c) 2007-2008 Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
+/**********************************************************************
+ * Copyright (C) 2000-2004 by Etnus, LLC.
+ * Copyright (C) 1999 by Etnus, Inc.
+ * Copyright (C) 1997-1998 Dolphin Interconnect Solutions Inc.
+ *
+ * Permission is hereby granted to use, reproduce, prepare derivative
+ * works, and to redistribute to others.
+ *
+ *				  DISCLAIMER
+ *
+ * Neither Dolphin Interconnect Solutions, Etnus LLC, nor any of their
+ * employees, makes any warranty express or implied, or assumes any
+ * legal liability or responsibility for the accuracy, completeness,
+ * or usefulness of any information, apparatus, product, or process
+ * disclosed, or represents that its use would not infringe privately
+ * owned rights.
+ *
+ * This code was written by
+ * James Cownie: Dolphin Interconnect Solutions. <jcownie@dolphinics.com>
+ *               Etnus LLC <jcownie@etnus.com>
+ **********************************************************************/
+
+#ifndef OMPI_MPIHANDLES_DLL_DEFS_H
+#define OMPI_MPIHANDLES_DLL_DEFS_H
+
+#include "ompi_common_dll_defs.h"
+
+/* JMS nothing specific yet */
+
+#endif

ompi/debuggers/ompi_msgq_dll_defs.h

@@ -1,3 +1,15 @@
+/*
+ * Copyright (c) 2007      Cisco, Inc.  All rights resereved.
+ * Copyright (c) 2004-2007 The University of Tennessee and The University
+ *                         of Tennessee Research Foundation.  All rights
+ *                         reserved.
+ * $COPYRIGHT$
+ * 
+ * Additional copyrights may follow
+ * 
+ * $HEADER$
+ */
+
 /**********************************************************************
  * Copyright (C) 2000-2004 by Etnus, LLC.
  * Copyright (C) 1999 by Etnus, Inc.
@@ -28,153 +40,10 @@
  *              info required by the DLL for dumping message queues.
  */
 
-/***********************************************************************
+#ifndef OMPI_MSGQ_DLL_DEFS_H
- * Information associated with a specific executable image
+#define OMPI_MSGQ_DLL_DEFS_H
- */
+
-typedef struct 
+#include "ompi_common_dll_defs.h"
-{
-    const struct mqs_image_callbacks * image_callbacks;	/* Functions needed here */
-    /* basic structures */
-    struct {
-        int size;
-        struct {
-            int opal_list_next;
-        } offset;
-    } opal_list_item_t;
-    struct {
-        int size;
-        struct {
-            int opal_list_sentinel;
-        } offset;
-    } opal_list_t;
-    struct {
-        int size;
-    } ompi_free_list_item_t;
-    struct {
-        int size;
-        struct {
-            int fl_elem_class;    /* opal_class_t* */
-            int fl_mpool;         /* struct mca_mpool_base_module_t* */
-            int fl_elem_size;     /* size_t */
-            int fl_alignment;     /* size_t */
-            int fl_allocations;   /* opal_list_t */
-            int fl_max_to_alloc;  /* size_t */
-            int fl_num_per_alloc; /* size_t */
-            int fl_num_allocated; /* size_t */
-        } offset;
-    } ompi_free_list_t;
-    /* requests structures */
-    struct {
-        int size;
-        struct {
-            int req_type;
-            int req_status;
-            int req_complete;
-            int req_state;
-            int req_f_to_c_index;
-        } offset;
-    } ompi_request_t;
-    struct {
-        int size;
-        struct {
-            int req_addr;
-            int req_count;
-            int req_peer;
-            int req_tag;
-            int req_comm;
-            int req_datatype;
-            int req_proc;
-            int req_sequence;
-            int req_type;
-            int req_pml_complete;
-        } offset;
-    } mca_pml_base_request_t;
-    struct {
-        int size;
-        struct {
-            int req_addr;
-            int req_bytes_packed;
-            int req_send_mode;
-        } offset;
-    } mca_pml_base_send_request_t;
-    struct {
-        int size;
-        struct {
-            int req_bytes_packed;
-        } offset;
-    } mca_pml_base_recv_request_t;
-#if 0
-    /* fragments for unexpected messages (as well as theirs headers) */
-    struct {
-        int size;
-        struct {
-            int hdr;
-            int request;
-        } offset;
-    } mca_pml_ob1_recv_frag_t;
-    struct {
-        int size;
-        struct {
-            int hdr_type;
-            int hdr_flags;
-        } offset;
-    } mca_pml_ob1_common_hdr_t;
-    struct {
-        int size;
-        struct {
-            int hdr_common;
-            int hdr_ctx;
-            int hdr_src;
-            int hdr_tag;
-            int hdr_seq;
-        } offset;
-    } mca_pml_ob1_match_hdr_t;
-#endif
-    /* communicator structures */
-    struct {
-        int size;
-        struct {
-            int lowest_free;
-            int number_free;
-            int size;
-            int addr;
-        } offset;
-    } opal_pointer_array_t;
-    struct {
-        int size;
-        struct {
-            int grp_proc_count;
-            int grp_my_rank;
-            int grp_flags;
-        } offset;
-    } ompi_group_t;
-    struct {
-        int size;
-        struct {
-            int c_name;
-            int c_contextid;
-            int c_my_rank;
-            int c_local_group;
-        } offset;
-    } ompi_communicator_t;
-    struct {
-        int size;
-        struct {
-            int MPI_SOURCE;
-            int MPI_TAG;
-            int MPI_ERROR;
-            int _count;
-            int _cancelled;
-        } offset;
-    } ompi_status_public_t;
-    struct {
-        int size;
-        struct {
-            int size;
-            int name;
-        } offset;
-    } ompi_datatype_t;
-} mpi_image_info; 
 
 /***********************************************************************
  * Information associated with a specific process
@@ -225,10 +94,7 @@ typedef struct {
  */
 typedef struct 
 {
-  const struct mqs_process_callbacks * process_callbacks; /* Functions needed here */
-
   struct communicator_t *communicator_list;	/* List of communicators in the process */
-  mqs_target_type_sizes sizes;			/* Process architecture information */
 
   /* Addresses in the target process */
   mqs_taddr_t send_queue_base;			/* Where to find the send message queues */
@@ -248,9 +114,6 @@ typedef struct
     
   mqs_ompi_free_list_t_pos next_msg;            /* And state for the message iterator */
   mqs_op_class  what;				/* What queue are we looking on */
-} mpi_process_info;
+} mpi_process_info_extra;
-
-
-
-
 
+#endif

ompi/mca/topo/base/topo_base_cart_coords.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -57,8 +58,7 @@ int mca_topo_base_cart_coords (MPI_Comm comm,
     for (i = 0; 
         (i < comm->c_topo_comm->mtc_ndims_or_nnodes) && (i < maxdims); 
         ++i, ++d) {
-
+        dim = *d;
-        dim = (*d > 0) ? *d : -(*d);
         remprocs /= dim;
         *coords++ = rank / remprocs;
         rank %= remprocs;

ompi/mca/topo/base/topo_base_cart_create.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -85,12 +86,6 @@ int mca_topo_base_cart_create (mca_topo_base_comm_t *topo_data,
        return MPI_SUCCESS;
    }
 
-   for (i = 0, p = topo_data->mtc_dims_or_index; i < ndims; ++i, ++p) {
-       *p = (*periods) ? -(*dims) : *dims;
-       ++dims;
-       ++periods;
-   }
-
    /* Have to replace this with the actual function body itself */
    p = topo_data->mtc_dims_or_index;
    coords =  topo_data->mtc_coords;
@@ -99,7 +94,7 @@ int mca_topo_base_cart_create (mca_topo_base_comm_t *topo_data,
    for (i=0; 
         (i < topo_data->mtc_ndims_or_nnodes && i < ndims); 
         ++i, ++p) {
-        dim = (*p > 0) ? *p : -(*p);
+        dim = *p;
         nprocs /= dim;
         *coords++ = dummy_rank / nprocs;
         dummy_rank %= nprocs;

ompi/mca/topo/base/topo_base_cart_get.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -17,6 +18,11 @@
  */
 
 #include "ompi_config.h"
+
+#ifdef HAVE_STRING_H
+#include <string.h>
+#endif
+
 #include "ompi/mca/topo/base/base.h"
 #include "ompi/communicator/communicator.h"
 #include "ompi/mca/topo/topo.h"
@@ -40,25 +46,14 @@ int mca_topo_base_cart_get (MPI_Comm comm,
                         int maxdims,
                         int *dims,
                         int *periods,
-                        int *coords){
+                        int *coords)
-    int i;
+{
-    int *d;
+    int m = (maxdims <= comm->c_topo_comm->mtc_ndims_or_nnodes) ?
-    int *c;
+        maxdims : comm->c_topo_comm->mtc_ndims_or_nnodes;
 
-    d = comm->c_topo_comm->mtc_dims_or_index;
+    memcpy(dims, comm->c_topo_comm->mtc_dims_or_index, m * sizeof(int));
-    c = comm->c_topo_comm->mtc_coords;
+    memcpy(periods, comm->c_topo_comm->mtc_periods_or_edges, m * sizeof(int));
-
+    memcpy(coords, comm->c_topo_comm->mtc_coords, m * sizeof(int));
-    for (i = 0; (i < comm->c_topo_comm->mtc_ndims_or_nnodes) && (i < maxdims); ++i) {
-         
-        if (*d > 0) {
-            *dims++ = *d++;
-            *periods++ = 0;
-        } else {
-           *dims++ = -(*d++);
-           *periods++ = 1;
-        }
-        *coords++ = *c++;
-    }
 
     return MPI_SUCCESS;
 }

ompi/mca/topo/base/topo_base_cart_rank.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -46,6 +47,7 @@ int mca_topo_base_cart_rank (MPI_Comm comm,
    int i;
    int *d;
    int *c;
+   int *p;
 
    /*
     * Loop over coordinates computing the rank.
@@ -55,13 +57,14 @@ int mca_topo_base_cart_rank (MPI_Comm comm,
 
     i = comm->c_topo_comm->mtc_ndims_or_nnodes - 1;
     d = comm->c_topo_comm->mtc_dims_or_index + i;
+    p = comm->c_topo_comm->mtc_periods_or_edges + i;
     c = coords + i;
 
-   for (; i >= 0; --i, --c, --d) {
+   for (; i >= 0; --i, --c, --d, --p) {
-       dim = (*d > 0) ? *d : -(*d);
+       dim = *d;
        ord = *c;
         if ((ord < 0) || (ord >= dim)) {
-          if (*d > 0) {
+          if (*p) {
              return MPI_ERR_ARG;
           }
           ord %= dim;

ompi/mca/topo/base/topo_base_cart_shift.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -52,7 +53,7 @@ int mca_topo_base_cart_shift (MPI_Comm comm,
     int srcord;
     int destord;
     int i;
-    int *p;
+    int *d, *q;
 
    /*
     * Handle the trivial case.
@@ -67,14 +68,11 @@ int mca_topo_base_cart_shift (MPI_Comm comm,
     * Compute the rank factor and ordinate.
     */
     factor = ompi_comm_size(comm);
-    p = comm->c_topo_comm->mtc_dims_or_index;
+    d = comm->c_topo_comm->mtc_dims_or_index;
-    for (i = 0; (i < comm->c_topo_comm->mtc_ndims_or_nnodes) && (i <= direction); ++i, ++p) {
+    q = comm->c_topo_comm->mtc_periods_or_edges;
-        if ((thisdirection = *p) > 0) {
+    for (i = 0; (i < comm->c_topo_comm->mtc_ndims_or_nnodes) && (i <= direction); ++i, ++d, ++q) {
-            thisperiod = 0;
+        thisdirection = *d;
-        } else {
+        thisperiod = *q;
-          thisperiod = 1;
-          thisdirection = -thisdirection;
-        }
 
         ord %= factor;
         factor /= thisdirection;

ompi/mca/topo/base/topo_base_cart_sub.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -72,7 +73,7 @@ int mca_topo_base_cart_sub (MPI_Comm comm,
      r = remain_dims + i;
 
      for (; i >= 0; --i, --d, --c, --r) {
-        dim = (*d > 0) ? *d : -(*d);
+        dim = *d;
         if (*r == 0) {
            colour += colfactor * (*c);
            colfactor *= dim;

ompi/mca/topo/unity/topo_unity.c

@@ -9,6 +9,7 @@
  *                         University of Stuttgart.  All rights reserved.
  * Copyright (c) 2004-2005 The Regents of the University of California.
  *                         All rights reserved.
+ * Copyright (c) 2008      Cisco Systems, Inc.  All rights reserved.
  * $COPYRIGHT$
  * 
  * Additional copyrights may follow
@@ -93,36 +94,17 @@ int mca_topo_unity_component_comm_unquery (struct ompi_communicator_t *comm)
 
 int mca_topo_unity_module_init (struct ompi_communicator_t *comm)
 {
-   /* This function is used to initialize the module on the communicator. We
+    /* Nothing to do -- the setup is done in communicator/comm.c
-    * need to hang the data off of the communicator. For this we still use the
+       (setup the comm->c_topo_comm data) */
-    * same data structure which was defined in topo.h, mca_topo_comm_1_0_0_t. 
-    * There are no additional things which we need. If some other module needs
-    * to hang additional data, then it has to have this structure as the first
-    * member and then extend. This is a must rule */
 
-   struct mca_topo_base_comm_1_0_0_t *topo_data;
-
-   /* allocate the data */
-
-   comm->c_topo_comm = NULL;
-   topo_data = (mca_topo_base_comm_1_0_0_t*)malloc(sizeof(struct mca_topo_base_comm_1_0_0_t));
-   if (NULL == topo_data) {
-       return OMPI_ERROR;
-   }
-
-   comm->c_topo_comm = topo_data;
     return OMPI_SUCCESS;
 }
 
    
 int mca_topo_unity_module_finalize (struct ompi_communicator_t *comm) 
 {
-    /* All we need to do for now is to remove the allocated data */
+    /* Nothing to do -- the teardown is done in
- 
+       communicator/comm_init.c (free the comm->c_topo_comm data) */
-    if (NULL != comm->c_topo_comm) {
-        free (comm->c_topo_comm);
-        comm->c_topo = NULL;
-    }
 
     return OMPI_SUCCESS;
 }

ompi/runtime/mpiruntime.h

@@ -102,11 +102,6 @@ int ompi_mpi_finalize(void);
 int ompi_mpi_abort(struct ompi_communicator_t* comm,
                    int errcode, bool kill_remote_of_intercomm);
 
-/**
- * Wait for a TotalView-like debugger if asked.
- */
-void ompi_mpi_wait_for_totalview(void);
-
 /**
  * Do a preconnect of MPI connections (i.e., force connections to
  * be made if they will be made).

ompi/runtime/ompi_mpi_init.c

@@ -752,8 +752,8 @@ int ompi_mpi_init(int argc, char **argv, int requested, int *provided)
                     ORTE_NAME_PRINT(ORTE_PROC_MY_NAME));
     }
 
-    /* Do we need to wait for a TotalView-like debugger? */
+    /* Do we need to wait for a debugger? */
-    ompi_wait_for_totalview();
+    ompi_wait_for_debugger();
 
     /* check for timing request - get stop time and report elapsed time if so */
     if (timing) {