 |
OpenMS
|
The macros of OpenMS' class-test framework (library OpenMSTestFramework), used by the test programs under src/tests/class_tests/.
More...
Macros | |
| #define | START_TEST(class_name, version) |
| Begin of the test program for a given class. | |
| #define | END_TEST |
| End of the test program for a class. | |
| #define | START_SECTION(name_of_test) |
| Begin of a subtest with a given name. | |
| #define | END_SECTION |
| End of a subtest. | |
| #define | TEST_EQUAL(a, b) |
| Generic equality macro. | |
| #define | TEST_TRUE(a) |
| Boolean test macro. | |
| #define | TEST_FALSE(a) |
| Boolean test macro. | |
| #define | TEST_NOT_EQUAL(a, b) |
| Generic inequality macro. | |
| #define | TEST_STRING_EQUAL(a, b) |
| std::string equality macro. | |
| #define | TEST_FILE_EQUAL(filename, templatename) |
| File comparison macro. | |
| #define | TEST_REAL_SIMILAR(a, b) |
| Floating point similarity macro. | |
| #define | TEST_STRING_SIMILAR(a, b) |
| std::string similarity macro. | |
| #define | TEST_FILE_SIMILAR(a, b) |
| File similarity macro. | |
| #define | TOLERANCE_RELATIVE(a) |
| Define the relative tolerance for floating point comparisons. | |
| #define | TOLERANCE_ABSOLUTE(a) |
| Define the absolute tolerance for floating point comparisons. | |
| #define | WHITELIST(a) TEST::setWhitelist(__FILE__, __LINE__, (a)); |
| Define the whitelist_ used by TEST_STRING_SIMILAR and TEST_FILE_SIMILAR. | |
| #define | TEST_EXCEPTION(exception_type, command) |
| Exception test macro. | |
| #define | TEST_PRECONDITION_VIOLATED(command) |
| Precondition test macro. | |
| #define | TEST_POSTCONDITION_VIOLATED(command) |
| Postcondition test macro. | |
| #define | TEST_EXCEPTION_WITH_MESSAGE(exception_type, command, message) |
| Exception test macro (with test for exception message). | |
| #define | NEW_TMP_FILE_EXT(filename, extension) |
| Create a temporary filename. | |
| #define | NEW_TMP_FILE(filename) filename = TEST::createTmpFileName(__FILE__, __LINE__); |
| #define | ABORT_IF(condition) |
| Skip the remainder of the current subtest. | |
| #define | STATUS(message) |
| Print a status message. | |
| #define | ADD_MESSAGE(message) |
| Sets an additional text that is displayed after final result of the test. | |
| #define | NOT_TESTABLE |
| Macro that suppresses the warning issued when no subtests are performed. | |
| #define | VALIDATE_TMP_FILES |
| Validate every file created with NEW_TMP_FILE against its XML schema. | |
| #define | VALIDATE_FILE(filename) |
| Validate a single file against its XML schema, at the point of the call. | |
Functions | |
| bool | validateTmpFiles (const std::vector< std::string > &file_names) |
| Validates the given files against their XML schema (if one applies). | |
The macros of OpenMS' class-test framework (library OpenMSTestFramework), used by the test programs under src/tests/class_tests/.
The framework lives in its own static library, OpenMSTestFramework (src/testframework/), and depends on the C++ standard library ONLY — it does not link or include libOpenMS. That is what lets the tests of every OpenMS library (including ones that do not link libOpenMS, such as OpenSwathAlgo) use the same macros, and it keeps the framework working unchanged when libOpenMS is split into smaller libraries.
A test program is one translation unit: START_TEST / END_TEST wrap the whole program, START_SECTION / END_SECTION wrap each subtest, and the elementary macros (TEST_EQUAL, TEST_REAL_SIMILAR, TEST_STRING_SIMILAR, TEST_FILE_EQUAL, TEST_FILE_SIMILAR, TEST_EXCEPTION, ...) record results. On success the program prints "PASSED", otherwise "FAILED" plus the failing lines. With -v it prints information about subsections, with -V (or the environment variable OPENMS_TEST_VERBOSE=True) about every elementary test.
Because the framework knows nothing about the library under test, library-specific behavior is registered by the test project (see src/tests/class_tests/source/OpenMSTestSupport.cpp, built as the OpenMSTestSupport target, which every class test of a project that links libOpenMS links in turn):
std::exception);Values in failure reports print via operator<< found at the macro-expansion site (so any type your test can see prints as usual); TEST_REAL_SIMILAR accepts floating point values and class types implicitly convertible to double (e.g. DataValue, ParamValue) — no registration needed.
Schema validation of temporary files is explicit: tests that write XML call VALIDATE_TMP_FILES (or VALIDATE_FILE) from OpenMS/TestFileValidation.h in the openms class-test project — it needs the FORMAT layer and therefore deliberately does not live in the framework.
To create a test, follow the guidelines in Developer FAQ (section "How to add a new class test"). Look at existing test files in src/tests/class_tests/ for examples.
| #define ABORT_IF | ( | condition | ) |
Skip the remainder of the current subtest.
If the condition is not fulfilled, the remainder of the current subtest is skipped over. The TEST status is set to FAIL.
| #define ADD_MESSAGE | ( | message | ) |
Sets an additional text that is displayed after final result of the test.
This can be used to provide additional information about the test to the user. It is e.g. used to indicate that the DB test were skipped, when there are no credentials given.
| #define END_SECTION |
End of a subtest.
The END_SECTION macro defines the end of a subtest.
Each elementary test macro updates an internal variable (TEST::test) that holds the state of the current subtest. END_SECTION prints whether the subtest has passed or failed (in verbose mode) and updates the internal variables TEST::all_tests that describes the state of the whole class test. TEST::all_tests is initialized to be true. If any elementary test fails, TEST::test becomes false. At the time of the next call to END_SECTION, TEST::all_tests will be set to false, if TEST::test is false. One failed elementary test leads therefore to a failed subtest, which leads to a failed class test.
This macro closes the try block opened by START_SECTION, so START_SECTION and END_SECTION have to be balanced, or some ugly compile-time errors will occur. END_SECTION catches any exception and reports it via the registered exception translators (see OpenMS::Internal::ClassTest::registerExceptionTranslator). After the exception is caught, the execution will continue with the next subtest, but the current subtest is marked as failed (as is the whole test program).
| #define END_TEST |
End of the test program for a class.
The END_TEST macro implements the correct termination of the test program and should therefore be the last macro to call. It determines the exit code based on all previously run subtests and prints out the message "PASSED" or "FAILED". This macro also closes the global try block opened by START_TEST and contains the related catch clauses. If an exception is caught here, the test program fails.
| #define NEW_TMP_FILE | ( | filename | ) | filename = TEST::createTmpFileName(__FILE__, __LINE__); |
| #define NEW_TMP_FILE_EXT | ( | filename, | |
| extension | |||
| ) |
Create a temporary filename.
This macro assigns a new temporary filename to the string variable given as its argument. The filename is created using the filename of the test and the line number where this macro is invoked, for example 'Matrix_test.cpp' might create a temporary file 'Matrix_test_268.tmp' if NEW_TMP_FILE is used in line 268. All temporary files are deleted if END_TEST is called. filename string will contain the filename on completion of the macro.
There is a version that defines the extension and one that uses tmp.
| #define NOT_TESTABLE |
Macro that suppresses the warning issued when no subtests are performed.
Please use this macro only if the method cannot be tested at all or cannot be tested properly on its own. In the later case, the method must however be tested in tests of related methods. See also test_count.
| #define START_SECTION | ( | name_of_test | ) |
Begin of a subtest with a given name.
The START_SECTION macro is used to declare the name of a subtest. Use this to examine a member function of the class which was specified in START_TEST. If you want to check e.g. the memFunc() method of a class, insert a line START_SECTION(memFunc()) in your test program. If the test program is called in verbose mode, this leads to the name of the subtest being printed on execution. If you are testing a non-public method you can use the [EXTRA] statement, e.g. START_SECTION([EXTRA]memFunc()) to mark a subtest that has no matching method declaration.
This macro also opens a try block to catch any unexpected exceptions thrown in the course of a subtest. To catch wanted exceptions (i.e. to check for exceptions that are the expected result of some command) use the TEST_EXCEPTION macro. The try block opened by START_SECTION is closed in END_SECTION, so these two macros have to be balanced.
| #define START_TEST | ( | class_name, | |
| version | |||
| ) |
Begin of the test program for a given class.
The START_TEST macro defines the start of the test program for a given classname. The classname is printed together with some information when calling the test program with any arguments (except for -v or -V).
The second argument version should take the form "$Id:$" but is currently deprecated. Originally, the SVN revision was annotated by the revision control system.
The START_TEST macro should be the first one to call in a test program. It opens a global try block to catch any unwanted exceptions. If any of these exceptions occurs, all tests fail. The report describes the caught exception via the registered exception translators (see OpenMS::Internal::ClassTest::registerExceptionTranslator), so e.g. OpenMS exceptions are reported with their name and origin. The END_TEST macro also closes the try block. This try block should never catch an exception! All exceptions that are thrown due to some malfunction in one of the member functions should be caught by the try block created by START_SECTION and END_SECTION .
| #define STATUS | ( | message | ) |
Print a status message.
If tests require longer preparations, STATUS may be used to print some intermediate progress messages. STATUS uses cout to print these messages (in verbose mode only). The given stream expression message is prefixed by the string status: and terminated with a newline. All valid operations on a stream may be performed in message.
Example: STATUS( "just calculated x = " << precisionWrapper(x) )
| #define TEST_EQUAL | ( | a, | |
| b | |||
| ) |
Generic equality macro.
This macro uses the operator == to check its two arguments for equality. Besides handling some internal stuff, it basically evaluates ((a) == (b)).
Remember that operator == has to be defined somehow for the two argument types. Additionally the << operator needs to be defined. If only == is available you will get a compilation error. As workaround use TEST_EQUAL(a==b, true) thereby making bug tracing harder, as you won't see the values of a and b.
| [in] | a | value/object to test |
| [in] | b | expected value |
| #define TEST_EXCEPTION | ( | exception_type, | |
| command | |||
| ) |
Exception test macro.
This macro checks if a given type of exception occurred while executing the given command. Example: TEST_EXCEPTION(Exception::IndexOverflow, vector[-1]). If no or a wrong exception occurred, false is returned, otherwise true.
| [in] | exception_type | the exception-class |
| [in] | command | any general C++ or OpenMS-specific command |
| #define TEST_EXCEPTION_WITH_MESSAGE | ( | exception_type, | |
| command, | |||
| message | |||
| ) |
Exception test macro (with test for exception message).
This macro checks if a given type of exception occurred while executing the given command and additionally tests for the message of the exception.
Example: TEST_EXCEPTION_WITH_MESSAGE(Exception::IndexOverflow, vector[-1], "a null pointer was specified")
If no, a wrong exception occurred or a wrong message is returned, false is returned, otherwise true.
| [in] | exception_type | the exception-class |
| [in] | command | any general C++ or OpenMS-specific command |
| [in] | message | the message the exception should give |
| #define TEST_FALSE | ( | a | ) |
Boolean test macro.
This macro tests if its argument evaluates to 'false'. If possible use TEST_NOT_EQUAL(a, b) instead of TEST_FALSE(a!=b), because the latter makes bug tracing harder.
| [in] | a | value/object convertible to bool |
| #define TEST_FILE_EQUAL | ( | filename, | |
| templatename | |||
| ) |
File comparison macro.
This macro is used to test file operations. It compares the file with name filename against a template file templatename. Corresponding lines of the two files have to be identical.
| #define TEST_FILE_SIMILAR | ( | a, | |
| b | |||
| ) |
File similarity macro.
Compares the two files using FuzzyStringComparator with the settings of TOLERANCE_ABSOLUTE and TOLERANCE_RELATIVE.
| [in] | a | value to test |
| [in] | b | expected value |
| #define TEST_NOT_EQUAL | ( | a, | |
| b | |||
| ) |
Generic inequality macro.
This macro checks for inequality just like TEST_EQUAL tests for equality. The only difference between the two macros is that TEST_NOT_EQUAL evaluates !((a) == (b)).
| [in] | a | value/object to test |
| [in] | b | forbidden value |
| #define TEST_POSTCONDITION_VIOLATED | ( | command | ) |
Postcondition test macro.
This macro checks if a postcondition violation is detected while executing the command, similar to TEST_EXCEPTION(Exception::Postcondition,command). However the test is executed only when the library's OPENMS_POSTCONDITION macros are active; see TEST_PRECONDITION_VIOLATED for how that is determined.
| [in] | command | any general C++ or OpenMS-specific command |
| #define TEST_PRECONDITION_VIOLATED | ( | command | ) |
Precondition test macro.
This macro checks if a precondition violation is detected while executing the command, similar to TEST_EXCEPTION(Exception::Precondition,command). However the test is executed only when the library's OPENMS_PRECONDITION macros are active; the test project registers that fact via setPreconditionTestsEnabled() (see OpenMSTestSupport.cpp), keyed on the same OPENMS_ASSERTIONS the library uses (see Macros.h).
| [in] | command | any general C++ or OpenMS-specific command |
| #define TEST_REAL_SIMILAR | ( | a, | |
| b | |||
| ) |
Floating point similarity macro.
Checks whether the two numbers are sufficiently close based upon the settings of TOLERANCE_ABSOLUTE and TOLERANCE_RELATIVE.
double. The actual comparison is done by isRealSimilar().| [in] | a | value to test |
| [in] | b | expected value |
| #define TEST_STRING_EQUAL | ( | a, | |
| b | |||
| ) |
std::string equality macro.
Both arguments are converted to std::string and tested for equality. (That is, we check whether (std::string(a) == std::string(b)) holds.)
| [in] | a | value to test |
| [in] | b | expected value |
| #define TEST_STRING_SIMILAR | ( | a, | |
| b | |||
| ) |
std::string similarity macro.
Compares the two strings using FuzzyStringComparator with the settings of TOLERANCE_ABSOLUTE and TOLERANCE_RELATIVE.
std::string. The actual comparison is done by testStringSimilar().| [in] | a | value to test |
| [in] | b | expected value |
| #define TEST_TRUE | ( | a | ) |
Boolean test macro.
This macro tests if its argument evaluates to 'true'. If possible use TEST_EQUAL(a, b) instead of TEST_TRUE(a==b), because the latter makes bug tracing harder.
| [in] | a | value/object convertible to bool |
| #define TOLERANCE_ABSOLUTE | ( | a | ) |
Define the absolute tolerance for floating point comparisons.
Several macros consider two numbers sufficiently "close" if the absolute difference is bounded by the value supplied by TOLERANCE_ABSOLUTE. The default value is \( 10^{-5} \). It is possible to redefine the absolute tolerance by calling TOLERANCE_ABSOLUTE with the new value.
| #define TOLERANCE_RELATIVE | ( | a | ) |
Define the relative tolerance for floating point comparisons.
Several macros consider two numbers sufficiently "close" if the ratio of the larger and the smaller is bounded by the value supplied by TOLERANCE_RELATIVE. The default value is \( 1 + 10^{-5} \). It is possible to redefine the relative tolerance by calling TOLERANCE_RELATIVE with the new value.
| #define VALIDATE_FILE | ( | filename | ) |
Validate a single file against its XML schema, at the point of the call.
Same checks as VALIDATE_TMP_FILES, but for one file and reported at this line, which makes a validation failure easier to attribute than a check at the end of the test. Also usable for files that were not created via NEW_TMP_FILE.
| #define VALIDATE_TMP_FILES |
Validate every file created with NEW_TMP_FILE against its XML schema.
Place this right before END_TEST in tests that write mzML, mzXML, mzData, featureXML, idXML, consensusXML, transformationXML or INI files. Files of any other type are skipped, so it is always safe to call.
Requires <OpenMS/TestFileValidation.h>. The test fails if any file is invalid.
| #define WHITELIST | ( | a | ) | TEST::setWhitelist(__FILE__, __LINE__, (a)); |
Define the whitelist_ used by TEST_STRING_SIMILAR and TEST_FILE_SIMILAR.
If both lines contain the same element from this list, they are skipped over. (See FuzzyStringComparator.)
|
inline |
Validates the given files against their XML schema (if one applies).
Files whose type is not one of the validatable XML formats are skipped.
| [in] | file_names | The files to check (usually TEST::tmp_file_list) |
References FileTypes::CONSENSUSXML, File::exists(), FileTypes::FEATUREXML, FileHandler::getType(), FileTypes::IDXML, FileTypes::INI, FileTypes::MZDATA, FileTypes::MZML, FileTypes::MZXML, FileTypes::TRANSFORMATIONXML, and FileTypes::typeToName().
1.9.8