921a29e41f
This commit adds mechanisms for writing and running unit tests in the usnic BTL. The short version of how to run the tests is: 1. Configure with `--enable-ompi-btl-usnic-unit-tests`. This will cause the unit testing code and test runner utility to be built. 2. Run the tests by running `ompi_btl_usnic_run_tests`. See `README.test` for full details. Reviewed-by: Jeff Squyres <jsquyres@cisco.com> cmr=v1.7.5:ticket=trac:4253 This commit was SVN r30845. The following Trac tickets were found above: Ticket 4253 --> https://svn.open-mpi.org/trac/ompi/ticket/4253
75 строки
3.5 KiB
Plaintext
75 строки
3.5 KiB
Plaintext
usnic BTL Unit Testing Information
|
|
==================================
|
|
This document briefly describes the scheme put in place for usnic BTL unit
|
|
testing. This system was very quickly tossed together and warrants a proper
|
|
revisiting at some point in the future. There are lots of ways to solve these
|
|
problems and this is just one of them. Future improvement is welcome.
|
|
|
|
Goals
|
|
-----
|
|
* To enable _unit_ testing of isolated functions or sets of functions. This
|
|
has all sorts of benefits, including:
|
|
- greater confidence that corner cases work as expected
|
|
- faster, lower-stress refactoring in the future
|
|
- influencing future interfaces to have less implicit coupling/state (unit
|
|
testing is harder otherwise)
|
|
* To be able to test *static* functions as well as non-static ones.
|
|
* To avoid cluttering the normal code base with excessive test-related
|
|
macros/code.
|
|
* The tests should be easy to run under Valgrind and GDB to facilitate:
|
|
- automated leak/memory checking
|
|
- easy debugging compared to a parallel MPI environment
|
|
|
|
Anti-Goals
|
|
----------
|
|
* Testing the low level networking API (e.g., verbs).
|
|
* Testing inter-process interaction, such as ORTE-related functionality.
|
|
|
|
Constraints
|
|
-----------
|
|
* our unit tests should never perturb a normal build in terms of performance
|
|
or correctness
|
|
- also should not affect other non-usNIC developers in any way (don't
|
|
break Ralph's `make check`)
|
|
* static functions are difficult to test from outside the same source file
|
|
|
|
Design Notes
|
|
------------
|
|
* Source files named `X.c` include a header at the end named `test/X_test.h`
|
|
- Rationale: gives tests access to the static functions in `X.c`
|
|
- Rationale: keeps `X.c` clutter-free
|
|
* unit test infrastructure lives in `btl_usnic_test.c` and `btl_usnic_test.h`
|
|
* unit test functionality is built and enabled by passing
|
|
`--enable-ompi-btl-usnic-unit-tests` to configure
|
|
- Rationale: default state disables all unit test logic, achieving our
|
|
"non-interference" goals
|
|
* The tests are run by a new executable that gets built when unit tests are
|
|
enabled: `ompi_btl_usnic_run_tests`.
|
|
* Tests are registered at dlopen time via an
|
|
`__attribute__((__constructor__))` function that is generated by invocations
|
|
of the `USNIC_REGISTER_TEST` macro.
|
|
- Rationale: add tests in one spot, no need to centralize the list of
|
|
tests to run separately from the tests themselves.
|
|
* Tests only use a simple `check()` macro right now that has `assert`-like
|
|
semantics.
|
|
- this could easily be expanded in the future, using the check docs as
|
|
inspiration:
|
|
http://check.sourceforge.net/doc/check_html/check_4.html#Convenience-Test-Functions
|
|
|
|
Worthy Future Goals/Features
|
|
----------------------------
|
|
* Add some mocking capabilities.
|
|
- could use the preprocessor to replace regular function/macro calls with
|
|
calls to functions/macros that dispatch to changeable function pointers
|
|
- will require some reorganization of some of the existing code...
|
|
* Output test results in a format that Jenkins and other tools can understand.
|
|
- TAP
|
|
- jUnit XML
|
|
* Possibly utilize part or all of an existing unit testing framework, e.g.:
|
|
- check: http://check.sourceforge.net/ (LGPLed)
|
|
- cUnit: http://cunit.sourceforge.net/ (unfortunately GPLed...)
|
|
* Re-examine test grouping and numbering. Right now there's no real concept
|
|
of "suites" or multiple cases within a single test function. Once the
|
|
number of tests grows to a certain point it will probably make sense to
|
|
revisit this decision.
|