1
1
Граф коммитов

163 Коммитов

Автор SHA1 Сообщение Дата
Ralph Castain
3ef94a0675 Per email thread on devel list:
Revert "OPAL: drop dead with core on bad flow. rarely happens with helloworld on large scale."

This reverts commit 86f1d5af3e.

Will be reconsidered via RFC as it represents a significant change in behavior
2014-10-12 21:13:42 -07:00
Ralph Castain
63f619f871 Provide a mechanism by which an upstream project can rename the OPAL and ORTE libraries. This is required by projects such as ORCM that have their own ORTE and OPAL libraries in order to avoid library confusion. By renaming their version of the libraries, the OMPI applications can correctly dynamically load the correct one for their build. 2014-10-10 11:39:08 -07:00
Ralph Castain
ce8e33447f Silence warning 2014-10-09 10:45:25 -07:00
Joshua Ladd
1cabd73522 Adding a new OPAL hash table routine. Please read the algorithm description in opal/class/opal_hash_table.c for more precise details on the design and implementation. This algorithm was contributed by David Linden of H.P. in partnership with Mellanox Technologies. This contribution achieves two objectives:
1. It's actually hashing now, whereas the old OPAL hash table was not. Thus, it is a bug fix for and, as such, should be included in the 1.8 series.

2. It is dynamic and can grow and shrink the number of buckets in accordance with job size, whereas the old OPAL hash table had a fixed number of buckets which resulted in poor retrieval performance at large scale.

This scheme has been deployed in the field on very large H.P./Mellanox systems and has been demonstrated to significantly decrease job start-up time (~ 20% improvement) when launching applications directly with srun in SLURM environments. However, neither SLURM nor direct launch are prerequisites to take advantage of this change as any entity that utilizes OPAL hash table objects can benefit (at least partially) from this contribution.
2014-10-09 17:24:23 +02:00
Jeff Squyres
a422d893b8 memchecker: per RFC, use calloc for OBJ_NEW
With --enable-memchecker builds, use calloc(3) for OBJ_NEW instead of
malloc(3).  This cuts down on a lot of valgrind/memory checker false
positive output.

Also make a minor change in the valgrind configure.m4; have it assign
0xf to a char.  The prior assignment (of 0xff) was warning about an
overflow.  This didn't really matter, but we might as well make the
test not have a gratuitious warning in it.
2014-10-07 09:55:54 -07:00
Mike Dubman
86f1d5af3e OPAL: drop dead with core on bad flow. rarely happens with helloworld on large scale. 2014-10-07 14:07:41 +03:00
Joshua Ladd
97abb7c727 Backing out the new Opal Hash table until the legal issues are address by H.P.
Refs trac:4872

This commit was SVN r32583.

The following Trac tickets were found above:
  Ticket 4872 --> https://svn.open-mpi.org/trac/ompi/ticket/4872
2014-08-22 19:10:09 +00:00
Gilles Gouaillardet
59caebe3ea new opal hash table
Decrease the hash table size when an element is removed

cmr=v1.8.2:ticket=trac:4872

This commit was SVN r32566.

The following Trac tickets were found above:
  Ticket 4872 --> https://svn.open-mpi.org/trac/ompi/ticket/4872
2014-08-21 06:51:11 +00:00
Ralph Castain
ea94659bd9 Silence warning
Refs trac:4872

This commit was SVN r32565.

The following Trac tickets were found above:
  Ticket 4872 --> https://svn.open-mpi.org/trac/ompi/ticket/4872
2014-08-20 22:27:03 +00:00
Joshua Ladd
84d0cc27a2 Adding a new OPAL hash table routine. Contributed by David Linden of H.P. in partnership
with Mellanox Technologies. This should be added to 


cmr=v1.8.2:subject=New OPAL hash table:reviewer=rhc

This commit was SVN r32564.
2014-08-20 21:40:28 +00:00
Gilles Gouaillardet
b5d5388c6c silence warning on solaris 10
use HAVE_STRINGS_H to protect <strings.h> include

cmr=v1.8.2:reviewer=rhc:ticket=trac:4853

This commit was SVN r32510.

The following Trac tickets were found above:
  Ticket 4853 --> https://svn.open-mpi.org/trac/ompi/ticket/4853
2014-08-12 04:27:49 +00:00
Gilles Gouaillardet
c28918c5cf silence warning on solaris 10
on solaris 10, bzero is declared in strings.h and not in string.h

cmr=v1.8.2:reviewer=rhc

This commit was SVN r32502.
2014-08-11 08:18:43 +00:00
Ralph Castain
552c9ca5a0 George did the work and deserves all the credit for it. Ralph did the merge, and deserves whatever blame results from errors in it :-)
WHAT:    Open our low-level communication infrastructure by moving all necessary components (btl/rcache/allocator/mpool) down in OPAL

All the components required for inter-process communications are currently deeply integrated in the OMPI layer. Several groups/institutions have express interest in having a more generic communication infrastructure, without all the OMPI layer dependencies.  This communication layer should be made available at a different software level, available to all layers in the Open MPI software stack. As an example, our ORTE layer could replace the current OOB and instead use the BTL directly, gaining access to more reactive network interfaces than TCP.  Similarly, external software libraries could take advantage of our highly optimized AM (active message) communication layer for their own purpose.  UTK with support from Sandia, developped a version of Open MPI where the entire communication infrastucture has been moved down to OPAL (btl/rcache/allocator/mpool). Most of the moved components have been updated to match the new schema, with few exceptions (mainly BTLs where I have no way of compiling/testing them). Thus, the completion of this RFC is tied to being able to completing this move for all BTLs. For this we need help from the rest of the Open MPI community, especially those supporting some of the BTLs.  A non-exhaustive list of BTLs that qualify here is: mx, portals4, scif, udapl, ugni, usnic.

This commit was SVN r32317.
2014-07-26 00:47:28 +00:00
George Bosilca
0b72cdabfd This is basically a revert of r32236. What looked as a good patch
turned out to be a nightmare, as the pointers to the classes are
located in shared libraries memory areas, and are not accesible
after the shared library is unloaded. Thus, OPAL cannot cleanup
the left-over classes from the other shared libraries.

This commit was SVN r32248.

The following SVN revision numbers were found above:
  r32236 --> open-mpi/ompi@59017433e1
2014-07-15 18:21:01 +00:00
George Bosilca
59017433e1 Allow for class allocation/deallocation.
This commit was SVN r32236.
2014-07-15 05:01:36 +00:00
Ralph Castain
811f3d0665 Remove unused vars and actually test the indexed array member in the bitmap class
This commit was SVN r32004.
2014-06-14 03:18:24 +00:00
George Bosilca
13252206ad Correctly handle the case where we restrict the size to INT_MAX.
This commit was SVN r32003.
2014-06-13 21:32:46 +00:00
George Bosilca
fbe69808f2 A faster implementation of the OPAL_BITMAP. The corresponding
test has also been updated.

This commit was SVN r32001.
2014-06-13 21:15:35 +00:00
Nathan Hjelm
0ef6baffd3 Fix bug in r31764
Need to remove the items of the list to avoid an assert in debug builds.

cmr=v1.8.2:ticket=trac:4628

This commit was SVN r31769.

The following SVN revision numbers were found above:
  r31764 --> open-mpi/ompi@13fd6ae774

The following Trac tickets were found above:
  Ticket 4628 --> https://svn.open-mpi.org/trac/ompi/ticket/4628
2014-05-14 23:45:50 +00:00
Nathan Hjelm
13fd6ae774 opal_free_list: destruct free lists items when destructing the free
list

This commit updates the behavior of opal_free_list_t to match the
behavior of ompi_free_list_t. opal_free_list_t constructed items
placed on the free list but never destructed them.

cmr=v1.8.2:reviewer=jsquyres

This commit was SVN r31764.
2014-05-14 21:15:19 +00:00
Ralph Castain
9514858067 As Rolf pointed out, this patch wasn't needed on the trunk - just the 1.7 branch. Sigh
This commit was SVN r30544.
2014-02-03 21:40:56 +00:00
Ralph Castain
4d533c81fb Minor cleanup required when configuring with an external libevent. Thanks to Orion Poplawski for the patch!
cmr=v1.7.4:reviewer=ompi-gk1.7

This commit was SVN r30543.
2014-02-03 21:03:05 +00:00
Jeff Squyres
9ba0d19ef1 Corrected patch from Tetsuya Mishima (i.e., a more correct fix than
r30086: make sure that a super item is constructed properly).

Refs trac:4035

This commit was SVN r30090.

The following SVN revision numbers were found above:
  r30086 --> open-mpi/ompi@d1c63f878e

The following Trac tickets were found above:
  Ticket 4035 --> https://svn.open-mpi.org/trac/ompi/ticket/4035
2013-12-26 12:41:39 +00:00
Ralph Castain
d1c63f878e Init variable to avoid infinite loop issues with PGI compilers
Thanks to Tetsuya Mishima for identifying the problem and providing the patch!

cmr=v1.7.4:reviewer=jsquyres:subject=Fix LAMA mapper for PGI compilers

This commit was SVN r30086.
2013-12-25 16:43:45 +00:00
Brian Barrett
6ef938de3f * Per the Developer's meeting today, restructure the threading in Open MPI a bit
more:
  - Remove OPAL_ENABLE_MULTI_THREADS, since it didn't really do anything
    correctly.  Opal always has threads enabled at this point.
  - Remove OMPI_ENABLE_PROGRESS_THREADS, since this hasn't worked in
    8 years and it has performance issues we'll never be able to
    overcome.  Note that we have plans for re-adding async progress, using
    a hybrid protocol of async and sync sends.
  - OMPI_ENABLE_THREAD_MULTIPLE now determines whether the thread lock
    macros do the check or not.
  - Condition variables are ALWAYS polling right now, which fixes the thread
    live-lock currently found when THREAD_MULTIPLE is turned on.

This commit was SVN r29891.
2013-12-13 19:40:12 +00:00
Jeff Squyres
bdb45a2e4f Add an oh-so-slightly faster variant of the hotel "checkin" action
(since this is used in the fast path) for when you ''know'' that there
will be a room available:

 * Don't do the last_unoccupied_room check
 * Return void

This commit was SVN r28757.
2013-07-11 20:00:37 +00:00
George Bosilca
a5bda43cfc Small typo.
This commit was SVN r28689.
2013-07-01 16:48:45 +00:00
George Bosilca
f5a55ccb39 Various cleanups.
This commit was SVN r28647.
2013-06-15 16:23:11 +00:00
Nathan Hjelm
8924140916 Per RFC: use a better hash algorithm for the opal_hash_table_*_ptr functions.
Chose the crc32 function present in opal/util/crc.c as the hash function. The
performance should be sufficient for most cases. If not we can always change
the function again.

This commit was SVN r28629.
2013-06-13 17:11:04 +00:00
Jeff Squyres
089c632cce Remove a bunch of dead code: gcc 4.7 warns of set-but-unused
variables.  So get rid of them.

This commit was SVN r28538.
2013-05-17 21:45:49 +00:00
Ralph Castain
c33219a51b Extend the bitmap API a bit to provide a test if all bits zero
This commit was SVN r28486.
2013-05-14 03:34:57 +00:00
George Bosilca
1169ebdff8 Indentation.
This commit was SVN r28426.
2013-04-30 23:26:23 +00:00
Ralph Castain
5d7a93c032 Add the ability to use an external version of libevent. Clearly not recommended at this time. I've verified that it works in limited scenarios, but more thorough testing and performance impacts need to be assessed.
Interesting how many includes had to be fixed here and there to fill in missing dependencies :-)

This commit was SVN r28411.
2013-04-29 17:02:37 +00:00
George Bosilca
a856f926de Remove a bunch of unused variables.
This commit was SVN r28213.
2013-03-26 14:34:29 +00:00
Ralph Castain
6dd4a8cdf9 The opal_list_t destructor doesn't release the items on the list prior to destructing or releasing it. Provide two convenience macros for doing so.
This commit was SVN r28029.
2013-02-04 19:42:57 +00:00
Nathan Hjelm
4bfb701115 add iterator macros for opal_list_t
This commit was SVN r27985.
2013-01-30 19:02:55 +00:00
Brian Barrett
fc3df11e08 Remove the (only two) fortran constants from OPAL. The only places that
actually care if opal_pointer_array is limited to handle_max already passes
that in as the max_size during init, so don't need it there.  The arch
constant was a bit more difficult, so pass that in during MPI init and
leave empty otherwise.

This is to help with the effort to allow building ompi against an external
opal or orte.

This commit was SVN r27817.
2013-01-15 01:27:36 +00:00
Jeff Squyres
3d05c5cca3 There's no point in having a separate opal_hotel_finalize() function
-- just move that functionality into the hotel destructor.

This commit was SVN r27555.
2012-11-02 14:00:54 +00:00
Jeff Squyres
e72c74a549 Fix backwards asserts in the OPAL hotel code.
This commit was SVN r27462.
2012-10-22 18:05:39 +00:00
Jeff Squyres
287e47a04d Fixes, improvements, and enhancements to the opal_tree class (used by
the LAMA RMAPS component, to be committed shortly).

This commit was SVN r27204.
2012-08-31 16:35:49 +00:00
Jeff Squyres
e497894c4d Gah!! We inlined some of the functionality, so we need these structs
to be defined.  Put comments in there indicating that they're private
and should not be used by public consumers.

This commit was SVN r27075.
2012-08-16 18:42:23 +00:00
Jeff Squyres
01256c36c6 Gah -- meant to make these changes before committing to SVN. :-\
Hide some struct declarations in the .c file to emphasize that they
are not part of the public opal_hotel interface.

This commit was SVN r27068.
2012-08-16 17:37:57 +00:00
Jeff Squyres
96f640a762 Add new "opal_hotel" class. Abstractly speaking, this class does the
following:

 * Provides a fixed number of resource slots (i.e., "hotel rooms").
 * Allows one thing to occupy a resource slot at a time (i.e., each
   hotel room can have an occupant check in to that room).
 * Resource slots can be vacated at any time (i.e., occupants can
   voluntarily check out of their hotel room).
 * Resource slots can be occupied for a specific maximum amount of
   time.  If that time expires, the occupant is forcibly evicted and
   the upper layer is notified via (libevent) callback (i.e., the maid
   will kick an occupant of out of their room when their reservation
   is over).

This class can be to be used for things like retransmission schemes
for unreliable transports.  For example, a message sent on an
unreliable transport can be checked in to a hotel room.  If an ACK for
that message is received, the message can be checked out.  But if the
ACK is never received, the message will eventually be evicted from its
room and the upper layer will be notified that the message failed to
check out in time (i.e., that an ACK for that message was not received
in time).

Code using this class is currently being developed off-trunk, but will
be coming to SVN soon.

This commit was SVN r27067.
2012-08-16 17:29:55 +00:00
George Bosilca
f7528bb404 Remove unused variables.
This commit was SVN r26966.
2012-08-08 12:43:13 +00:00
George Bosilca
2303cd0bdb Remove initialized but unused variables.
This commit was SVN r26959.
2012-08-07 12:05:25 +00:00
Josh Hursey
f7c66f54b6 Add a pair of functions to count the number of set/unset bits in an opal_bitmap.
I found this useful on a branch, and thought others might as well.

This commit was SVN r26171.
2012-03-21 14:16:45 +00:00
George Bosilca
2589e55a75 item_in_tree is only used in debug mode, so protect it.
This commit was SVN r25548.
2011-11-29 23:48:26 +00:00
Rainer Keller
4e6a6fc146 - Check, whether the compiler supports __builtin_clz (count leading
zeroes);
   if so, use it for bit-operations like opal_cube_dim and opal_hibit.
   Implement two versions of power-of-two.
   In case of opal_next_poweroftwo, this reduces the average execution
   time from 83 cycles to 4 cycles (Intel Nehalem, icc, -O2, inlining,
   measured rdtsc, with loop over 2^27 values).
   Numbers for other functions are similar (but of course heavily depend
   on the usage, e.g. opal_hibit() with a start of 4 does not save
   much).  The bsr instruction on AMD Opteron is also not as fast.

 - Replace various places where the next power-of-two is computed.
   
   Tested on Intel Nehalem Cluster with openib, compilers GNU-4.6.1 and
   Intel-12.0.4 using mpi_testsuite -t "Collective" with 128 processes.

This commit was SVN r25270.
2011-10-11 22:49:01 +00:00
Ralph Castain
9244ea10fb Provide a way to look at the head of the ring
This commit was SVN r24832.
2011-06-28 19:46:48 +00:00
Josh Hursey
8cd5280299 Some assorted opal_bitmap extensions
* Protect the '->bitmap' field if init() is called more than once [it shouldn't be, but if it is then this avoids a memory leak].
 * Some new functions
   * opal_bitmap_bitwise_and_inplace
   * opal_bitmap_bitwise_or_inplace
   * opal_bitmap_bitwise_xor_inplace
   * opal_bitmap_are_different
   * opal_bitmap_get_string

Adding these features to the trunk so others have access to them if they need them. A couple off trunk branches make use of them.

This commit was SVN r24767.
2011-06-09 14:43:54 +00:00