...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
Unless specified otherwise, when a value of floating-point type is compared
inside a BOOST_TEST
assertion, operators
==
, !=
, <
etc. defined for this
type are used. However for floating point type, in most cases what is needed
is not an exact equality (or inequality), but a verification
that two numbers are sufficiently close or sufficiently
different. For that purpose, a tolerance
parameter that will instruct the framework what is considered sufficiently
close needs to provided.
Note | |
---|---|
How the tolerance parameter is processed in detail is described here. |
It is possible to define a per-test unit
tolerance for a given floating point type by using decorator
tolerance
:
Code |
---|
#define BOOST_TEST_MODULE tolerance_01 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant difference BOOST_TEST(x == z); // relevant difference } |
Output |
---|
> tolerance_01 Running 1 test case... test.cpp(11): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 1 failure is detected in the test module "tolerance_01" |
It is possible to specify floating point comparison tolerance per single
assertion, by providing the manipulator boost::test_tools::tolerance
as the second argument to BOOST_TEST
:
Code |
---|
#define BOOST_TEST_MODULE tolerance_02 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant by default BOOST_TEST(x == y, tt::tolerance(0.0)); BOOST_TEST(x == z); // relevant by default BOOST_TEST(x == z, tt::tolerance(0.001)); } |
Output |
---|
> tolerance_02 Running 1 test case... test.cpp(12): error: in "test1": check x == y has failed [10 != 10.000000099999999] test.cpp(14): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 2 failures are detected in the test module "tolerance_02" |
Caution | |
---|---|
Manipulators requires a compiler that supports variadic macros, |
It is possible to specify the tolerance as percentage. At test unit level, the decorator syntax is:
* boost::unit_test::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) ) // equivalent to: boost::unit_test::tolerance( 2.0 / 100 )
At assertion level, the manipulator syntax is:
2.0% boost::test_tools::tolerance() boost::test_tools::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) ) // both equivalent to: boost::test_tools::tolerance( 2.0 / 100 )
Manipulator tolerance
specifies
the tolerance only for a single floating-point type. This type is deduced
from form the numeric value passed along the manipulator:
expression |
semantics |
---|---|
|
tolerance for type |
|
tolerance for type |
|
tolerance for type |
|
tolerance for type |
|
tolerance for a user-defined type |
|
tolerance for type |
|
tolerance for type |
|
tolerance for type |
This is also the case for decorator tolerance
.
In the case of the decorator however, it is possible to apply multiple
decorators tolerance
defining
the tolerance for different types.
When values of two different floating point types T
and U
are compared, BOOST_TEST
uses the tolerance
specified for type boost::common_type<T, U>::type
. For instance, when setting a tolerance
for mixed float
-to-double
comparison, the tolerance for type
double
needs to be set.
Given two floating point types T
and U
and their common
type C
, the tolerance specified
for type C
is applied only
when types T
and U
appear as sub-expressions of the full
expression inside assertion BOOST_TEST
. It is not applied
when T
and U
are compared inside a function invoked
during the evaluation of the expression:
Code |
---|
#define BOOST_TEST_MODULE tolerance_05 #include <boost/test/included/unit_test.hpp> #include <boost/optional.hpp> #include <boost/optional/optional_io.hpp> BOOST_AUTO_TEST_CASE(test, * boost::unit_test::tolerance(0.02)) { double d1 = 1.00, d2 = 0.99; boost::optional<double> o1 = 1.00, o2 = 0.99; BOOST_TEST(d1 == d2); // with tolerance (double vs. double) BOOST_TEST(o1 == o2); // without tolerance (optional vs. optional) BOOST_TEST(o1 == d2); // without tolerance (optional vs. double) BOOST_TEST(*o1 == *o2); // with tolerance (double vs. double) } |
Output |
---|
> tolerance_05 Running 1 test case... ../doc/examples/tolerance_05.run.cpp(20): error: in "test": check o1 == o2 has failed [ 1 != 0.99] ../doc/examples/tolerance_05.run.cpp(21): error: in "test": check o1 == d2 has failed [ 1 != 0.98999999999999999] *** 2 failures are detected in the test module "tolerance_05" |
Given two types T
and
U
being compared inside
an assertion BOOST_TEST
, tolerance based comparison
is invoked
T
and U
are both tolerance
based types
T
is tolerance
based and U
is arithmetic,
in the sense that std::numeric_limits<U>::value
evaluates to true
(or
the other way round)
In all cases, the type of the tolerance is deduced as boost::common_type<T, U>::type
, and both type may be cast to this
tolerance type.
Note | |
---|---|
This behaviour has been introduced in Boost 1.70 / Unit Test Framework 3.10. Previously tolerance based comparison was used only when the type of the two operands were tolerance based types, which was silently ignoring the tolerance for expressions such as double x = 1E-9; BOOST_TEST(x == 0); // U is int |
Code |
---|
#define BOOST_TEST_MODULE tolerance_06 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.1415 / 3)) // == 0.047166667 { double x = 3.141592404915836; // x is 'double' which is tolerance based, 3 is 'int' which is arithmetic: // tolerance based comparison will be used. // Type of tolerance for this comparison will be boost::common_type<double, int>::type == double // Value for this tolerance type is set by the decorator. BOOST_TEST(x == 3); } |
Output |
---|
> tolerance_06 Running 1 test case... test.cpp(21): error: in "test1": check x == 3 has failed [3.1415924049158361 != 3]. Relative difference exceeds tolerance [0.0471975 > 0.0471667] *** 1 failure is detected in the test module "tolerance_06" |
Finally, note that comparisons for tolerance are also applied to operator<
with semantics less by more than some tolerance, and
other relational operators. Also, the tolerance-based comparisons are involved
when a more complicated expression tree is processed within the assertion
body. The section on relational
operators defines how operator<
relates to tolerance.
Code |
---|
#define BOOST_TEST_MODULE tolerance_03 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; double x = 10.000000; double d = 0.000001; BOOST_AUTO_TEST_CASE(passing, * utf::tolerance(0.0001)) { BOOST_TEST(x == x + d); // equal with tolerance BOOST_TEST(x >= x + d); // ==> greater-or-equal BOOST_TEST(d == .0); // small with tolerance } BOOST_AUTO_TEST_CASE(failing, * utf::tolerance(0.0001)) { BOOST_TEST(x - d < x); // less, but still too close BOOST_TEST(x - d != x); // unequal but too close BOOST_TEST(d > .0); // positive, but too small BOOST_TEST(d < .0); // not sufficiently negative } |
Output |
---|
> tolerance_03 Running 2 test cases... test.cpp(18): error: in "failing": check x - d < x has failed [10 - 1.0000000000000001e-05 >= 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(19): error: in "failing": check x - d != x has failed [10 - 1.0000000000000001e-05 == 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(21): error: in "failing": check d > .0 has failed [1.0000000000000001e-05 <= 0]. Absolute value is within tolerance [|1e-05| < 0.001] test.cpp(22): error: in "failing": check d < .0 has failed [1.0000000000000001e-05 >= 0] *** 4 failures are detected in the test module "tolerance_03" |