1
1
Граф коммитов

4001 Коммитов

Автор SHA1 Сообщение Дата
Nathan Hjelm
bccf8c657a Per RFC add initial support for the MPI 3.0 tools interface.
Current MPI_T support:
  - Full cvar interface.
  - Full categories interface.
  - No pvar support at this time.

This commit was SVN r28376.
2013-04-24 15:59:23 +00:00
Ralph Castain
2e8946db0a Add some debug output
This commit was SVN r28371.
2013-04-23 23:11:22 +00:00
Ralph Castain
b6377a2138 Properly remove the object from the list prior to releasing it when an error is encountered
This commit was SVN r28370.
2013-04-23 22:44:52 +00:00
Ralph Castain
1a54e52da4 Per conversation with Josh H., remove the stale rsh component from the filem framework. Let the "raw" component do the job.
This commit was SVN r28338.
2013-04-16 20:39:48 +00:00
Ralph Castain
d6ac721e22 Add client/server test
This commit was SVN r28332.
2013-04-15 13:10:42 +00:00
Jeff Squyres
349ee654c1 Fix some --without-hwloc compile errors. Also remove one
assigned-but-not-used variable assignment.

This commit was SVN r28321.
2013-04-10 15:08:31 +00:00
Ralph Castain
45af6cf59e The move of the orte_db framework to opal required that we create an opaque opal_identifier_t type as OPAL cannot know anything about the ORTE process name. However, passing a value down to opal and then having the db components reference it causes alignment issues on Solaris Sparc platforms. So pass the pointer instead and do the old "memcpy" trick to avoid the problem.
This commit was SVN r28308.
2013-04-08 23:34:16 +00:00
Ralph Castain
2040afdcae Daemonize prior to starting any progress threads
This commit was SVN r28303.
2013-04-08 00:42:57 +00:00
Ralph Castain
76426285f0 Cannot retain opal_buffer_t, so use a copy
This commit was SVN r28302.
2013-04-07 23:02:59 +00:00
Ralph Castain
698b4ad6e7 Fix the parameter handling so no-tree-spawn isn't getting reversed
This commit was SVN r28300.
2013-04-07 15:48:25 +00:00
Ralph Castain
1f011bef99 Cleanup the updated sys limits capability. Fix a few copy/paste bugs (my bad). Shift the limit set to the ODLS default module so that we sete the limits for all apps, even those that don't call opal_init. Leave it in opal_init as well to support direct-launch apps, but ensure we only set the limits once by removing the envar after launch by ODLS.
Provide some nice error messages if we fail to set the limits. Since the user had to specifically request we set the limit, treat failure as an error-out situation.

This commit was SVN r28288.
2013-04-04 16:00:17 +00:00
Ralph Castain
252147fba6 Cleanup error message if unknown host is given in -host and -hostfile options
This commit was SVN r28262.
2013-03-28 16:52:10 +00:00
Ralph Castain
e6ae088813 Cleanup error outputs when a daemon fails to start
This commit was SVN r28261.
2013-03-28 16:51:19 +00:00
Ralph Castain
21ee48de57 Add missing static declaration
This commit was SVN r28247.
2013-03-27 21:59:17 +00:00
Ralph Castain
1b5b9afa85 Silence warnings
This commit was SVN r28244.
2013-03-27 21:56:46 +00:00
Nathan Hjelm
17315bf360 Now that the entire codebase has been updated to use the MCA framework
system remove the last calls to the MCA parameter system.

This commit was SVN r28242.
2013-03-27 21:17:53 +00:00
Nathan Hjelm
9d4a26f47d Update OMPI frameworks to use the MCA framework system.
Notes:
  - This commit also eliminates the need for an available components list in use
    in several frameworks. None of the code in question was making use of the
    priority field of the priority component list item so these extra lists were
    removed.
  - Cleaned up selection code in several frameworks to sort lists using opal_list_sort.
  - Cleans up the ompi/orte-info functions. Expose the functions that construct the
    list of params so they can be used elsewhere.

patches for mtl/portals4 from brian

missed a few output variables in openib

This commit was SVN r28241.
2013-03-27 21:17:31 +00:00
Nathan Hjelm
c041156f60 Update ORTE frameworks to use the MCA framework system.
This commit was SVN r28240.
2013-03-27 21:14:43 +00:00
Nathan Hjelm
365cf48db5 Update OPAL frameworks to use the MCA framework system.
This commit was SVN r28239.
2013-03-27 21:11:47 +00:00
Nathan Hjelm
c3b67d0187 Automatically generate a list of installed frameworks in project/include/project/frameworks.h
This commit was SVN r28238.
2013-03-27 21:10:32 +00:00
Nathan Hjelm
cf377db823 MCA/base: Add new MCA variable system
Features:
 - Support for an override parameter file (openmpi-mca-param-override.conf).
   Variable values in this file can not be overridden by any file or environment
   value.
 - Support for boolean, unsigned, and unsigned long long variables.
 - Support for true/false values.
 - Support for enumerations on integer variables.
 - Support for MPIT scope, verbosity, and binding.
 - Support for command line source.
 - Support for setting variable source via the environment using
   OMPI_MCA_SOURCE_<var name>=source (either command or file:filename)
 - Cleaner API.
 - Support for variable groups (equivalent to MPIT categories).

Notes:
 - Variables must be created with a backing store (char **, int *, or bool *)
   that must live at least as long as the variable.
 - Creating a variable with the MCA_BASE_VAR_FLAG_SETTABLE enables the use of
   mca_base_var_set_value() to change the value.
 - String values are duplicated when the variable is registered. It is up to
   the caller to free the original value if necessary. The new value will be
   freed by the mca_base_var system and must not be freed by the user.
 - Variables with constant scope may not be settable.
 - Variable groups (and all associated variables) are deregistered when the
   component is closed or the component repository item is freed. This
   prevents a segmentation fault from accessing a variable after its component
   is unloaded.
 - After some discussion we decided we should remove the automatic registration
   of component priority variables. Few component actually made use of this
   feature.
 - The enumerator interface was updated to be general enough to handle
   future uses of the interface.
 - The code to generate ompi_info output has been moved into the MCA variable
   system. See mca_base_var_dump().

opal: update core and components to mca_base_var system
orte: update core and components to mca_base_var system
ompi: update core and components to mca_base_var system

This commit also modifies the rmaps framework. The following variables were
moved from ppr and lama: rmaps_base_pernode, rmaps_base_n_pernode,
rmaps_base_n_persocket. Both lama and ppr create synonyms for these variables.

This commit was SVN r28236.
2013-03-27 21:09:41 +00:00
Ralph Castain
e7ac6c9bde Don't build rank_file if you can't use it anyway
This commit was SVN r28233.
2013-03-27 15:12:40 +00:00
Ralph Castain
256414121e Protect the cpus-per-rank MCA param registration so that --without-hwloc will build
This commit was SVN r28232.
2013-03-27 14:53:30 +00:00
Ralph Castain
317915225c Finish the binding cleanup by removing the no-longer-used binding level scheme. This proved to be fallible as there is no guarantee that the hierarchy it used matched physical reality of the machine (e.g., is L3 "above" the socket or not). Still have to complete the ppr update, but get the rest of it correct.
This commit was SVN r28223.
2013-03-26 20:09:49 +00:00
Ralph Castain
24b91839aa Ensure the process knows it local cpuset early enough to perform the locality computation
This commit was SVN r28221.
2013-03-26 19:14:23 +00:00
Ralph Castain
6ee32767d4 Restore the cpus-per-proc option for byslot and bynode mapping. Remove the bind_idx (which recorded the index of the hwloc object where the proc was bound) as this would no longer be unique, and just use the bitmap as the standard reference for location. Update the relative locality computation to take bitmaps as its argument.
This commit was SVN r28219.
2013-03-26 18:27:50 +00:00
Ralph Castain
9c68e60965 Add test for comm_spawn with info keys
This commit was SVN r28207.
2013-03-24 14:38:33 +00:00
Ralph Castain
2f43989d22 Add debug and handle the use-case where someone (a) uses a hostfile while in a managed allocation to sub-allocate runs, and (b) includes the HNP's node in one of those hostfiles.
cmr:v1.7

This commit was SVN r28203.
2013-03-22 00:53:33 +00:00
Jeff Squyres
63d17ce901 Fix CID 968581: ensure that the string read from the socket is always
\0-terminated so that strlen() and strstr() can be used without fear.
Also fix some insignificant mem leaks (which is somewhat moot, because
as soon as we leave those error conditions, the process will be
terminating, but what the heck, might as well fix these while I was in
the file for the \0-termination issue...).

This commit was SVN r28199.
2013-03-21 16:05:50 +00:00
Jeff Squyres
562db0dd11 Fix CID 741328: remove some dead code
This commit was SVN r28192.
2013-03-21 11:15:06 +00:00
Ralph Castain
fa13d27238 Avoid double-release in error path
This commit was SVN r28190.
2013-03-20 21:00:59 +00:00
Ralph Castain
147c6ff9e7 Clean out the cruft leftover from the use_common_ports experiment
cmr:v1.7

This commit was SVN r28184.
2013-03-20 15:07:43 +00:00
Ralph Castain
a4b6fb241f Remove all remaining vestiges of the Windows integration
This commit was SVN r28137.
2013-02-28 17:31:47 +00:00
Ralph Castain
63727aa714 Use a non-blocking send in show_help as it could be called from inside an event
This commit was SVN r28135.
2013-02-28 17:19:18 +00:00
Ralph Castain
cf9796accd Remove the old configure option for disabling full rte support - we now use the OMPI rte framework for such purposes
This commit was SVN r28134.
2013-02-28 01:35:55 +00:00
Ralph Castain
347df93cd4 Handle the case of someone specifying a directory for the application. Ensure we get a non-zero exit status and clarify the error message.
cmr:v1.7

This commit was SVN r28119.
2013-02-27 01:36:21 +00:00
Ralph Castain
f36312ee6f Continue cleanup - this time, start working on the "without full support" flags in ORTE. Remove no-longer-needed configure.m4 files from the ess and errmgr. In the former case, since all priorities are now the same (given the removal of the cnos component), configure priorities are no longer required.
This commit was SVN r28118.
2013-02-26 21:27:48 +00:00
Ralph Castain
74a3ece313 Remove unused component
This commit was SVN r28117.
2013-02-26 20:58:43 +00:00
Ralph Castain
8d2fa3693b First cut at removing the native Windows support. Remove all the Windows-specific components, and the .windows files sprinkled around. Remove the Windows platform files and MTT scripts. Update the NEWS to point Windows users to the cygwin package.
This commit was SVN r28116.
2013-02-26 20:44:56 +00:00
Ralph Castain
bd9265c560 Per the meeting on moving the BTLs to OPAL, move the ORTE database "db" framework to OPAL so the relocated BTLs can access it. Because the data is indexed by process, this requires that we define a new "opal_identifier_t" that corresponds to the orte_process_name_t struct. In order to support multiple run-times, this is defined in opal/mca/db/db_types.h as a uint64_t without identifying the meaning of any part of that data.
A few changes were required to support this move:

1. the PMI component used to identify rte-related data (e.g., host name, bind level) and package them as a unit to reduce the number of PMI keys. This code was moved up to the ORTE layer as the OPAL layer has no understanding of these concepts. In addition, the component locally stored data based on process jobid/vpid - this could no longer be supported (see below for the solution).

2. the hash component was updated to use the new opal_identifier_t instead of orte_process_name_t as its index for storing data in the hash tables. Previously, we did a hash on the vpid and stored the data in a 32-bit hash table. In the revised system, we don't see a separate "vpid" field - we only have a 64-bit opaque value. The orte_process_name_t hash turned out to do nothing useful, so we now store the data in a 64-bit hash table. Preliminary tests didn't show any identifiable change in behavior or performance, but we'll have to see if a move back to the 32-bit table is required at some later time.

3. the db framework was a "select one" system. However, since the PMI component could no longer use its internal storage system, the framework has now been changed to a "select many" mode of operation. This allows the hash component to handle all internal storage, while the PMI component only handles pushing/pulling things from the PMI system. This was something we had planned for some time - when fetching data, we first check internal storage to see if we already have it, and then automatically go to the global system to look for it if we don't. Accordingly, the framework was provided with a custom query function used during "select" that lets you seperately specify the "store" and "fetch" ordering.

4. the ORTE grpcomm and ess/pmi components, and the nidmap code,  were updated to work with the new db framework and to specify internal/global storage options.

No changes were made to the MPI layer, except for modifying the ORTE component of the OMPI/rte framework to support the new db framework.

This commit was SVN r28112.
2013-02-26 17:50:04 +00:00
Jeff Squyres
9bd4b814db Fix one more nroff macro issue
This commit was SVN r28090.
2013-02-21 17:38:06 +00:00
Jeff Squyres
76fcd42bc3 Fix minor nroff macro issues.
This commit was SVN r28088.
2013-02-21 17:35:36 +00:00
Jeff Squyres
12e047e594 Update documentation for rankfiles in orterun.1:
* Add a little more description of what rankfiles are
 * Update that we use logical numbering for socket:core notation
 * Mention +nX notation

This commit was SVN r28067.
2013-02-16 17:52:30 +00:00
Ralph Castain
c0b670bea8 I guess some profiling tools and debuggers require that the argv[0] of each rank be unique so they can create a filename based on that value. For those obscure cases, provide an mpirun cmd line option that indexes each argv[0] by rank
This commit was SVN r28064.
2013-02-15 20:20:49 +00:00
Ralph Castain
b9897267ef Cleanup report-bindings so it always reports the actual binding instead of what was requested. Ensure we don't report twice if it is an MPI process being launched.
This commit was SVN r28057.
2013-02-14 17:24:28 +00:00
Ralph Castain
744ed49b2d Begin cleanup of the thread_lock calls in ORTE. We'll ignore the ones in the rml/oob for now as that code block is being rewritten anyway.
This commit was SVN r28053.
2013-02-13 01:53:12 +00:00
Brian Barrett
504a6d036f * Rather than use the extra_includes directive, add the extra includes (which is really just -I${includedir}/openmpi/ for devel headers) to CPPFLAGS, since all the other necessary -Is for devel headers (like libevent and hwloc) are added to CPPFLAGS.
* Clean up ${includedir} and ${libdir} for script wrapper compilers
* Update script wrapper compilers to work like the C wrapper compilers w.r.t static and dynamic linking
* Remove the ORTE script wrapper compilers since they didn't support the ${includedir} stuff and Ralph said they weren't used anymore.

This commit was SVN r28052.
2013-02-13 00:33:05 +00:00
Joshua Ladd
70ad711337 Backing out the Open SHMEM project
This commit was SVN r28050.
2013-02-12 17:45:27 +00:00
Mike Dubman
ff384daab4 Added new project: oshmem.
This commit was SVN r28048.
2013-02-12 15:33:21 +00:00
Ralph Castain
b360156a37 Extend print coverage to all types
This commit was SVN r28012.
2013-02-01 14:21:06 +00:00
Ralph Castain
afb0db5b6f Okay, Jeff - just for you...flow the show help thru the orte functions so help messages will be aggregated
This commit was SVN r28007.
2013-02-01 00:35:48 +00:00
Ralph Castain
53e0ed71b0 Disqualify slurm module even if slurm support was configured into the build if we don't have an allocation and haven't enabled dynamic allocations
This commit was SVN r27995.
2013-01-31 18:15:47 +00:00
Ralph Castain
166f512924 Add some useful debug to the heartbeat sensor
This commit was SVN r27994.
2013-01-31 18:01:13 +00:00
Ralph Castain
ca9605773b If sensors are enabled, then the daemons need to have their proc->node field linked to their local node object
This commit was SVN r27991.
2013-01-31 16:38:57 +00:00
Ralph Castain
c87fa68f9b Cleanup the resource usage sensor, letting the db handle any printing requests.
This commit was SVN r27990.
2013-01-31 15:20:56 +00:00
Ralph Castain
9625757a71 Add new database component for printing "add_log" info
This commit was SVN r27989.
2013-01-31 15:19:39 +00:00
Ralph Castain
8e8e95ca6b Silence error report - just because someone only defines ipv4 static ports doesn't make a fatal error
This commit was SVN r27976.
2013-01-29 23:48:22 +00:00
Jeff Squyres
8e25b927ab Clean some minor warnings: remove variables that were set but never
used.

This commit was SVN r27974.
2013-01-29 23:35:42 +00:00
Ralph Castain
112f8eedb1 Handle the case where rankfile is providing the allocation
This commit was SVN r27971.
2013-01-29 20:37:58 +00:00
Nathan Hjelm
666bd826dc fix alps configury
This commit was SVN r27962.
2013-01-29 15:44:30 +00:00
Brian Barrett
b8442ba505 Revamp the handling of wrapper compiler flags. The user flags, main configure
flags, and mca flags are kept seperate until the very end.  The main configure
wrapper flags should now be modified by using the OPAL_WRAPPER_FLAGS_ADD
macro.  MCA components should either let <framework>_<component>_{LIBS,LDFLAGS}
be copied over OR set <framework>_<component>_WRAPPER_EXTRA_{LIBS,LDFLAGS}.
The situations in which WRAPPER CPPFLAGS can be set by MCA components was
made very small to match the one use case where it makes sense.

This commit was SVN r27950.
2013-01-29 00:00:43 +00:00
Ralph Castain
cfaefb3286 Remove the only place where PMI was used outside a component, and relocate that code to common/pmi.
This commit was SVN r27944.
2013-01-28 20:14:51 +00:00
Brian Barrett
f42783ae1a Move the RTE framework change into the trunk. With this change, all non-CR
runtime code goes through one of the rte, dpm, or pubsub frameworks.

This commit was SVN r27934.
2013-01-27 23:25:10 +00:00
Ralph Castain
6eaf601ae6 Good ol' Cray changed the way node/cpu allocation is handled in their latest release of ALPS, and so our allocator is broken. Adjust for the revised method, but preserve the older method for those Cray users who have not updated their system.
cmr:v1.7

This commit was SVN r27911.
2013-01-25 21:53:31 +00:00
Ralph Castain
f6b4db0b79 Fix rank_file operations. We changed the syntax to use semi-colons between multiple slot assignments so that we could use the comma to separate specific cores, but somehow the flex definitions didn't get updated to accept that character. We also incorrectly zero'd the bitmap between slot assignment sections, and so multiple slot assignments only wound up making the last one in the list.
This commit was SVN r27908.
2013-01-25 18:33:25 +00:00
Ralph Castain
2504da1ac9 Remove stale code - message arrival time doesn't really mean much anymore.
This commit was SVN r27905.
2013-01-24 23:02:02 +00:00
Brian Barrett
0e799a93c3 Automake will ship the .in file whether or not the conditional is taken,
so don't install orte_wrapper_script when it's not used

This commit was SVN r27902.
2013-01-24 21:36:25 +00:00
Ralph Castain
9bfb2b989b Silence warning
This commit was SVN r27901.
2013-01-24 19:38:51 +00:00
Ralph Castain
4b310473a1 Correct the computation of the daemon vpid
cmr:v1.7

This commit was SVN r27899.
2013-01-24 18:04:53 +00:00
Ralph Castain
b403ca5bd8 Silence warning
This commit was SVN r27897.
2013-01-23 22:17:08 +00:00
Ralph Castain
4d34d30a97 Silence warning
This commit was SVN r27896.
2013-01-23 22:16:48 +00:00
Ralph Castain
6e2cabb87f Remove duplicate code
This commit was SVN r27889.
2013-01-23 02:07:06 +00:00
Ralph Castain
a591fbf06f Add initial support for dynamic allocations. At this time, only Slurm supports the new capability, which will be included in an upcoming release.
Add hooks for supporting dynamic allocation and deallocation to support application-driven requests and fault recovery operations.

This commit was SVN r27879.
2013-01-20 00:33:42 +00:00
Ralph Castain
e4673f3283 Add new job state
This commit was SVN r27878.
2013-01-20 00:30:27 +00:00
Ralph Castain
7aa80b984d Add new test program
This commit was SVN r27877.
2013-01-20 00:29:45 +00:00
Ralph Castain
7102d7c5f7 ick - brain is fried. take that test out as it isnt needed on a regular basis
This commit was SVN r27875.
2013-01-19 14:48:31 +00:00
Ralph Castain
38786457cb Add new test
This commit was SVN r27874.
2013-01-19 14:46:23 +00:00
Ralph Castain
73387e50e2 Add missing variable def - thanks to Paul Hargrove for spotting.
This commit was SVN r27865.
2013-01-18 14:32:53 +00:00
George Bosilca
e69dc00460 Dont duplicate headers nor global variables.
This commit was SVN r27864.
2013-01-18 11:51:56 +00:00
Ralph Castain
c96cc2d5a0 In order to properly connect to debuggers like STAT, we need to get the hostname in its unstripped version for the MPIR_proctab. Unfortunately, we need a stripped version for Cray's alps launcher. So when we are stripping the hostname prefix, retain alias hostnames and add the ability to specify an alias to use in the proctab.
This commit was SVN r27863.
2013-01-18 05:00:05 +00:00
Ralph Castain
54266837e9 Remove use of param_find function as that function will be disappearing
This commit was SVN r27831.
2013-01-15 19:50:38 +00:00
Ralph Castain
5b8de0b9f4 Ouch - opal_progress calls event_loop with a NO_BLOCK flag. So when run without progress threads, the ORTE tools were not blocking in the event lib as they should be. Avoid calling opal_progress inside ORTE by directly using the event_loop call instead of ORTE_WAIT_FOR_COMPLETION as parts of the OMPI layer are using that macro.
Thanks to George for spotting the problem.

This commit was SVN r27815.
2013-01-14 23:06:42 +00:00
Ralph Castain
97ee683275 Track parentage of procs
This commit was SVN r27758.
2013-01-08 04:41:12 +00:00
Ralph Castain
aea6787918 Add new routed component with self-healing connections - based on radix component - for use in monitoring system
This commit was SVN r27757.
2013-01-08 04:40:35 +00:00
Ralph Castain
c9a596b487 Remove unused var
This commit was SVN r27756.
2013-01-08 04:39:30 +00:00
Ralph Castain
beddf3b379 Add required rml tag
This commit was SVN r27751.
2013-01-05 06:32:20 +00:00
Ralph Castain
bee8bf5d8f Update the sensor framework to report stats back to the HNP if requested by including the data in heartbeats.
This commit was SVN r27748.
2013-01-05 06:30:20 +00:00
Ralph Castain
c71e119bbb Extend the db framework to add support for logging data to databases without duplicating all the modex-related storage.
This commit was SVN r27746.
2013-01-05 06:28:09 +00:00
George Bosilca
34eecb8956 Be more explicit about the operation (store or update). complain loudly
if something goes wrong.

This commit was SVN r27743.
2013-01-04 20:47:25 +00:00
Ralph Castain
cc29f8ff95 Attempt to fix the stupid Cray PMI problem
This commit was SVN r27742.
2013-01-04 02:53:42 +00:00
Nathan Hjelm
6a9ab9b221 Change orte_startup_timeout to be in seconds and remove the 10 second maximum
This commit was SVN r27741.
2013-01-03 23:56:34 +00:00
Ralph Castain
81a8e21939 Need to have the event thread running during init/finalize, but we still have a problem with cleanup - so comment out the event_base_free for now.
This commit was SVN r27738.
2013-01-03 02:16:57 +00:00
Ralph Castain
c65de32218 Cleanup the PMI subsystems to support Sam's "rml-less" shared memory wireup. Only retrieve keys that are specifically requested, and only when they are requested. Let string values be segmented across multiple keys, but don't do it for anything else.
This commit was SVN r27737.
2013-01-03 02:16:10 +00:00
Ralph Castain
c1690f403e Remove non-existent file
This commit was SVN r27730.
2012-12-29 02:21:50 +00:00
Ralph Castain
68329b516c Cleanup stale test codes
This commit was SVN r27729.
2012-12-28 16:52:51 +00:00
Ralph Castain
d1163ebbf2 Ensure we cleanup DFS worker threads during finalize to avoid segfaulting in MCA param cleanup
This commit was SVN r27723.
2012-12-25 21:17:35 +00:00
Ralph Castain
64da742d5f Remove the orte_finalize_event variable - no longer needed
This commit was SVN r27722.
2012-12-25 19:33:20 +00:00
Ralph Castain
cada035f38 Fix the segfault problem in the orteds - turns out it only occurred with progress threads enabled. Ensure the thread gets started at the right time (at the end of init), although the event base gets created earlier. Remove the finalize event as we can instead use the loopbreak call to exit the event loop.
This commit was SVN r27721.
2012-12-25 19:30:18 +00:00
Ralph Castain
c8e34813b6 THIS IS A TEMPORARY FIX - do not finalize opal as the parameter system has been broken and will segfault when finalized.
THIS PATCH MUST BE REMOVED WHEN THE PARAMETER SYSTEM HAS BEEN FIXED.

This commit was SVN r27720.
2012-12-24 18:42:19 +00:00
Ralph Castain
72bea688f1 Fix typo
This commit was SVN r27717.
2012-12-23 18:13:39 +00:00
Ralph Castain
852a709c0e Add libopen-pal to the libraries as all these tools directly reference OPAL functions, and the list of OS's that don't support indirect linking grows (Mac and Ubuntu, for now).
This commit was SVN r27716.
2012-12-23 15:54:05 +00:00
Jeff Squyres
b29b852281 Consolidate all the opal/orte/ompi .m4 files back to the top-level
config/ directory.  We split them apart a while ago in the hopes that
it would simplify things, but it didn't really (e.g., because there
were still some ompi/opal .m4 files in the top-level config/
directory, resulting in developer confusion where any given m4 macro
was defined).

So this commit consolidates them back into the top-level directory for
simplicity.  

There's still (at least) two changes that would be nice to make:

 1. Split any generated .m4 file (e.g., autogen-generated .m4 files)
    into a separate directory somewhere so that a top-level -Iconfig/
    will only get our explicitly defined macros, not the autogen stuff
    (e.g., with libevent2019 needing to get the visibility macro, but
    NOT all the autogen-generated inclusion of component configure.m4
    files).
 1. Change configure to be of the form:
{{{
# ...a small amount of preamble/setup...
OPAL_SETUP
m4_ifdef([project_orte], [ORTE_SETUP])
m4_ifdef([project_ompi], [OMPI_SETUP])
# ...a small amount of finishing stuff...
}}}

I doubt we'll ever get anything as clean as that, but that would be
the goal to shoot for.

This commit was SVN r27704.
2012-12-19 00:00:36 +00:00
Ralph Castain
ab73d11368 Oops - push missing definitions
This commit was SVN r27688.
2012-12-18 16:43:03 +00:00
Ralph Castain
c5ba59ba67 Remove stale component
This commit was SVN r27684.
2012-12-18 04:01:16 +00:00
Ralph Castain
0427a478b2 Remove stale component
This commit was SVN r27683.
2012-12-18 04:00:51 +00:00
Ralph Castain
82f1ba0ea8 Fix static port usage, ensure that both ipv4 and ipv6 are given if ipv6 was enabled
This commit was SVN r27682.
2012-12-18 03:59:49 +00:00
Ralph Castain
2fdd367aa9 Refs trac:3429
Fix bug reported by FreyGuy19713: in cases where HNP node has multiple entries in a hostfile or other allocation, we need to track the total slots allocated to that node.

This commit was SVN r27673.

The following Trac tickets were found above:
  Ticket 3429 --> https://svn.open-mpi.org/trac/ompi/ticket/3429
2012-12-14 17:00:44 +00:00
Jeff Squyres
c5b0bcd9f7 Refs trac:3422
* Add some comments in the *-wrapper-data-txt.in files just so that
   someone doesn't forget in the future why we link in what we do in
   the MPI and ORTE wrapper compilers.
 * Update ompi_wrapper_script.in to match the new behavior.
 * Update orte_wrapper_script.in to support --openmpi:linkall (which
   is a no-op in this case)

This commit was SVN r27672.

The following Trac tickets were found above:
  Ticket 3422 --> https://svn.open-mpi.org/trac/ompi/ticket/3422
2012-12-14 16:34:20 +00:00
Jeff Squyres
f779b1ded9 Put back the static-library-detection stuff from r27668, with some
additional functionality.  Rationale (refs trac:3422):

 * Normal MPI applications only ever use the MPI API. Hence, -lmpi is
   sufficient (they'll never directly call ORTE or OPAL
   functions). This is arguably the most common case.
 * That being said, we do have some test programs (e.g., those in
   orte/test/mpi) that call MPI functions but also call ORTE/OPAL
   functions. I've also written the occasional MPI test program that
   calls opal_output, for example (there even might be a few tests in
   the IBM test suite that directly call ORTE/OPAL functions).
   * Even though this is not a common case, these applications should
     also compile/link with mpicc.
   * So we should add a --openmpi:linkall option that will also link
     in whatever is necessary to call ORTE/OPAL functions
   * Yes, we could hard-code "-lopen-rte -lopen-pal" in Makefiles, but
     we do reserve the right to change those library names and/or add
     others someday, so it's better to abstract out the names and let
     the wrapper supply whatever is necessary.
 * ORTE programs, however, are different. They almost always call OPAL
   functions (e.g., if they want to send a message, they must use the
   OPAL DSS). As such, it seems like the ORTE programs should always
   link in OPAL.

Therefore:

 * Add undocumented --openmpi:linkall flag to the wrapper compilers.
   See the comment in opal_wrapper.c for an explanation of what it
   does.  This flag is only intended for Open MPI developers -- not
   end users.  That's why it's undocumented.
 * Update orte/test/mpi/Makefile.am to add --openmpi:linkall
 * Make ortecc/ortec++'s wrapper data text files always explicitly
   link in libopen-pal

This commit was SVN r27670.

The following SVN revision numbers were found above:
  r27668 --> open-mpi/ompi@cf845897aa

The following Trac tickets were found above:
  Ticket 3422 --> https://svn.open-mpi.org/trac/ompi/ticket/3422
2012-12-13 22:31:37 +00:00
Jeff Squyres
cf845897aa Temporarily revert r27662 and r27667 because something wonky is
happening on OS X.  Grumble...

This commit was SVN r27668.

The following SVN revision numbers were found above:
  r27662 --> open-mpi/ompi@97cc916007
  r27667 --> open-mpi/ompi@529f6244ca
2012-12-11 23:08:14 +00:00
Jeff Squyres
97cc916007 Per discussion at the Open MPI developer meeting last week:
1. Restore libopen-pal.la, libopen-rte.la, and libmpi.la to be
    separate entities (i.e., don't have libopen-rte.la include
    libopen-pal.la, and don't have libmpi.la include libopen-pal.la).
    Yay!
 1. Consequently, make the wrapper compilers look for flags indicating
    that the user wants to compile statically (currently: -static,
    !--static, -Bstatic, and "-Wl," in front of all of those).  If it
    is, follow a 6-way matrix for determinining which libraries to
    list on the underlying command line.
 1. To support that, add the name of a token static and dynamic
    library to look for in each of the wrapper compiler data files.
 1. Fix a long-standing typo in the opalcc wrapper data file.

This commit was SVN r27662.
2012-12-11 01:46:59 +00:00
Ralph Castain
1e92aa2b66 Enable multiple worker threads for processing DFS requests
This commit was SVN r27659.
2012-12-09 02:54:19 +00:00
Ralph Castain
c26ed7dcdd Fix comm_spawn when ORTE progress thread is enabled by ensuring that all operations on the global list of active collectives are done in events to avoid conflicts.
This commit was SVN r27658.
2012-12-09 02:53:20 +00:00
Nathan Hjelm
3e1b13b13a Re-add support for old flex (2.5.4a and earlier) while still cleaning up properly in new flex.
This commit was SVN r27657.
2012-12-07 00:12:43 +00:00
Ralph Castain
1237f8db57 Extend the ras module interface to include the orte_job_t being allocated so that dynamic allocations can be supported
This commit was SVN r27627.
2012-11-23 13:50:10 +00:00
George Bosilca
994d1aba50 Nothing.
This commit was SVN r27626.
2012-11-21 20:07:20 +00:00
Nathan Hjelm
a427a7e727 do not include c99 flag in compiler wrappers
This commit was SVN r27625.
2012-11-20 19:33:14 +00:00
Ralph Castain
7a5f6b584c Have orte-info show thread support as well
This commit was SVN r27624.
2012-11-18 18:15:22 +00:00
Ralph Castain
43f883cb42 Add some more detailed error output to the db_hash component and nidmap code. Ensure the local nodename is included in the HNP's aliases
This commit was SVN r27622.
2012-11-18 17:57:19 +00:00
Ralph Castain
f2ec35536e Fix a bug that prevented MCA params from being forwarded to daemons upon launch
cmr:v1.7

This commit was SVN r27621.
2012-11-18 17:55:26 +00:00
Ralph Castain
e11f32038a Add an MCA param to retain all aliases based on IP addrs for node names so that procs can look them up by interface, if desired. If the param is set, pass aliases around to all daemons and procs for local use
This commit was SVN r27619.
2012-11-16 04:04:29 +00:00
Ralph Castain
da6428a822 Add an MCA param to help debug the ORTE progress thread
This commit was SVN r27614.
2012-11-15 15:54:38 +00:00
Ralph Castain
5241925b62 Add the dfs to ompi_info
This commit was SVN r27613.
2012-11-15 15:54:07 +00:00
Ralph Castain
3cecc1569b Fix segfault if no file_maps were pushed
This commit was SVN r27612.
2012-11-15 15:39:17 +00:00
Ralph Castain
fe6dfad625 Update DFS to support multi-node operations
This commit was SVN r27594.
2012-11-12 02:54:53 +00:00
Ralph Castain
fefec03e78 Enable all ORTE tools to use progress threads if they are enabled
This commit was SVN r27593.
2012-11-12 02:54:09 +00:00
Ralph Castain
a6325e4546 Silence compiler warning
This commit was SVN r27590.
2012-11-12 02:51:29 +00:00
Ralph Castain
26f1cd0909 Fix compiler warnings
This commit was SVN r27588.
2012-11-12 02:50:45 +00:00
Ralph Castain
bd887f7f56 Add a new "test" component to the DFS that treats all files as remote in order to test the app-to-daemon interactions on a single machine. Set a global param to indicate we are using staged execution. Add a param to indicate it is okay for non-MPI processes to execute without finalizing. Cleanup file map load and fetch operations.
This commit was SVN r27587.
2012-11-10 14:09:12 +00:00
Ralph Castain
81d0b06842 Strip the domain info from the hostname if that option is specified, protecting IP address-based names
This commit was SVN r27586.
2012-11-10 14:05:27 +00:00
Ralph Castain
615cc66b44 Protect the HNP cleanup in cases where no session dirs are created
This commit was SVN r27585.
2012-11-10 14:03:07 +00:00
Ralph Castain
fd632147df Per patch from Nathan, with a few fixes, cleanup the orte-info tool
This commit was SVN r27581.
2012-11-10 04:11:40 +00:00
Nathan Hjelm
e0f5137e46 add prototypes for lex destroy functions
This commit was SVN r27580.
2012-11-09 22:00:27 +00:00
Nathan Hjelm
a754674fd7 Per the specification for putenv (http://pubs.opengroup.org/onlinepubs/009604599/functions/putenv.html) the string given to putenv becomes part of the environment. The string must not be changed or freed.
cmr:v1.7

This commit was SVN r27578.
2012-11-09 16:33:14 +00:00
Nathan Hjelm
8658bbc902 instead of relying on yyterminate to clean up the lex context call the destroy functions directly (after closing the file)
This commit was SVN r27577.
2012-11-09 16:10:55 +00:00
Nathan Hjelm
842caae4c7 Fix a small leak in orte/util/name_fns.c
cmr:v1.7

This commit was SVN r27576.
2012-11-07 23:59:49 +00:00
Ralph Castain
9b729794f2 A prior commit apparently broke the trunk when something was inadvertently left behind - so remove a reference to a no-longer-existing function
This commit was SVN r27574.
2012-11-07 11:11:05 +00:00
Nathan Hjelm
7fb5caea92 Remove the finish_parsing function from various .l files. The function is incomplete (doesn't clean up the lex state) and should be replaced by *_yylex_destroy which correctly cleans up the state.
Checked with the flex 2.5.35. Verified with valgrind that this fixes several "still reachable" leaks.

cmr:v1.7

This commit was SVN r27571.
2012-11-06 19:26:14 +00:00
Nathan Hjelm
bdedd8b0d3 Per RFC modify the behavior of mca_base_components_close to NOT close the output. Modify frameworks to always close their output and set to -1.
Reasoning: The old behavior was a little confusing. mca_base_components_open does not open an output stream so it is a little unexpected that mca_base_components_close does. To add to this several frameworks (that don't use mca_base_components_close) failed to close their output in the framework close function and others closed their output a second time. This change is an improvement to the symantics of mca_base_components_open/close as they are now symetric in their functionality.

This commit was SVN r27570.
2012-11-06 19:09:26 +00:00
Ralph Castain
27b41a7db4 If the nodename is an IP address, we need to retain the full name (even if keep_fqdn is false) so that the ssh tree spawn can proceed.
cmr:v1.7

This commit was SVN r27561.
2012-11-05 16:59:53 +00:00
Ralph Castain
3812979315 Remove stale file - we removed the size functions awhile back
This commit was SVN r27551.
2012-11-01 03:36:51 +00:00
Brian Barrett
e61c00212d Add files found in svn but not tarball
This commit was SVN r27549.
2012-11-01 02:27:03 +00:00
Brian Barrett
dd907e8d4c Remove unused macro
This commit was SVN r27547.
2012-11-01 02:04:57 +00:00
Nathan Hjelm
2acd0f83de Revert "Revert r27451 and r27456 - the cmd line parser is incorrectly marking the application as an MCA parameter".
It appears the problem was not with the command line parser but the rsh plm. I don't know why this problem was not occuring before the command line parser changes but it appears to be resolved now.

This commit was SVN r27527.

The following SVN revision numbers were found above:
  r27451 --> open-mpi/ompi@d59034e6ef
  r27456 --> open-mpi/ompi@ecdbf34937
2012-10-30 19:45:18 +00:00
Nathan Hjelm
df9bd0ed59 fix bug in plm/rsh that could add extraneous mca options to the orted argv
cmr:v1.7

This commit was SVN r27526.
2012-10-30 19:40:04 +00:00
Ralph Castain
7c3a2c3c92 Allow PMI support to find subdirs named lib64 instead of lib - thanks to Guillaume.Papaure of Bull for the patch
This commit was SVN r27519.
2012-10-30 17:38:57 +00:00
Ralph Castain
a080de188f Enable orterun to directly support staged execution, treating each app as a separate job. Support transfer of file maps when support exists.
This commit was SVN r27516.
2012-10-29 23:11:30 +00:00
Ralph Castain
e5e72c3137 Expand the dfs API to support retrieval, loading and purging of file maps.
This commit was SVN r27515.
2012-10-29 23:05:45 +00:00
Ralph Castain
4e52a15e70 Provide for sync on seek and close DFS operations. Eliminate an unnecessary wake-up timer when using ORTE progress thread
This commit was SVN r27500.
2012-10-26 15:49:04 +00:00
Ralph Castain
4ef30c016b Remove stale windows references
This commit was SVN r27491.
2012-10-26 01:19:14 +00:00