5295902ebe
* allow receive_queues to be specified in the INI file
* detect when multiple different receive_queues are specified and
gracefully abort
However, accomplishing these goals ran into multiple difficulties. By
putting receive_queues in the INI file:
1. we may not find the value until we've already traversed multiple HCAs
1. we may find multiple different receive_queues values
But since the openib btl initializes as it discovers each HCA/port/LID
(including the BSRQ data), if we find a new receive_queues value late
in the discovery process, then all the BSRQ data that was previously
initialized will likely be invalid. So I had to pull all the BSRQ
initialization out until after the rest of the discovery /
initialization process.
Additionally, note that if the user specifies the MCA parameter
btl_openib_receive_queues, it trumps whatever was in the INI file. So
in this case, there can never be a receive_queues conflict. This
commit does the following (Jon wrote part of this, too):
* adapt _ini.c to accept the "receive_queues" field in the file
* move 90% of _setup_qps() from _ini.c to _component.c
* move what was left of _setup_qps() into the main
_register_mca_params() function
* adapt init_one_hca() to detect conflicting receive_queues values
from the INI file
* after the _component.c loop calling init_one_hca():
* call setup_qps() to parse the final receive_queues string value
* traverse all resulting btls and initialize their HCAs (if they
weren't already): setup some lists and call prepare_hca_for_use()
I tested this code on a dual-HCA system where I artificially put in
differing receive_queues values in the INI file for the two different
types of HCAs that I have and it all seemed to work.
This commit was SVN r18450.
The following Trac tickets were found above:
Ticket 1285 --> https://svn.open-mpi.org/trac/ompi/ticket/1285
59 строки
1.1 KiB
C
59 строки
1.1 KiB
C
/*
|
|
* Copyright (c) 2006 Cisco Systems, Inc. All rights reserved.
|
|
* $COPYRIGHT$
|
|
*
|
|
* Additional copyrights may follow
|
|
*
|
|
* $HEADER$
|
|
*
|
|
* @file
|
|
*/
|
|
|
|
#ifndef MCA_PTL_IB_PARAMS_H
|
|
#define MCA_PTL_IB_PARAMS_H
|
|
|
|
#include "btl_openib.h"
|
|
|
|
|
|
/*
|
|
* Struct to hold the settable values that may be specified in the INI
|
|
* file
|
|
*/
|
|
typedef struct ompi_btl_openib_ini_values_t {
|
|
uint32_t mtu;
|
|
bool mtu_set;
|
|
|
|
uint32_t use_eager_rdma;
|
|
bool use_eager_rdma_set;
|
|
|
|
char *receive_queues;
|
|
bool receive_queues_set;
|
|
} ompi_btl_openib_ini_values_t;
|
|
|
|
|
|
#if defined(c_plusplus) || defined(__cplusplus)
|
|
extern "C" {
|
|
#endif
|
|
|
|
/**
|
|
* Read in the INI files containing HCA params
|
|
*/
|
|
int ompi_btl_openib_ini_init(void);
|
|
|
|
/**
|
|
* Query the read-in params for a given HCA
|
|
*/
|
|
int ompi_btl_openib_ini_query(uint32_t vendor_id,
|
|
uint32_t vendor_part_id,
|
|
ompi_btl_openib_ini_values_t *values);
|
|
|
|
/**
|
|
* Shut down / release all internal state
|
|
*/
|
|
int ompi_btl_openib_ini_finalize(void);
|
|
|
|
#if defined(c_plusplus) || defined(__cplusplus)
|
|
}
|
|
#endif
|
|
#endif
|