Floating point comparison

Enabling tolerance for user-defined types
Tolerance-based comparisons
Theory behind floating point comparisons

Unless you specify otherwise, when two values of floating-point type are compared inside assertion BOOST_TEST, operators == and != defined for these types are used. In most cases, however, what we need is not an exact equality (or inequality), but to check that the two numbers are 'sufficiently close' or 'sufficiently different'. In order to do that we need to provide a tolerance parameter that will instruct the framework what we consider 'sufficiently close'.

We can define a per-test unit tolerance for a given floating point type by using decorator tolerance:

Example: specifying tolerance per test case

Code
#define BOOST_TEST_MODULE tolerance_01 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant difference BOOST_TEST(x == z); // relevant difference }

Code

#define BOOST_TEST_MODULE tolerance_01
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;

BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001))
{
  double x = 10.0000000;
  double y = 10.0000001;
  double z = 10.001;
  BOOST_TEST(x == y); // irrelevant difference
  BOOST_TEST(x == z); // relevant difference
}

Output
> tolerance_01 Running 1 test case... test.cpp(11): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 1 failure is detected in the test module "tolerance_01"

Output

> tolerance_01
Running 1 test case...
test.cpp(11): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005]

*** 1 failure is detected in the test module "tolerance_01"

How the tolerance parameter is processed in detail is described here.

It is possible to specify floating point comparison tolerance per single assertion, by providing manipulator boost::test_tools::tolerance as the second argument to BOOST_TEST.

Example: specifying tolerance per assertion

Code
#define BOOST_TEST_MODULE tolerance_02 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001)) { double x = 10.0000000; double y = 10.0000001; double z = 10.001; BOOST_TEST(x == y); // irrelevant by default BOOST_TEST(x == y, tt::tolerance(0.0)); BOOST_TEST(x == z); // relevant by default BOOST_TEST(x == z, tt::tolerance(0.001)); }

Code

#define BOOST_TEST_MODULE tolerance_02
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;
namespace tt = boost::test_tools;

BOOST_AUTO_TEST_CASE(test1, * utf::tolerance(0.00001))
{
  double x = 10.0000000;
  double y = 10.0000001;
  double z = 10.001;
  BOOST_TEST(x == y); // irrelevant by default
  BOOST_TEST(x == y, tt::tolerance(0.0));

  BOOST_TEST(x == z); // relevant by default
  BOOST_TEST(x == z, tt::tolerance(0.001));
}

Output
> tolerance_02 Running 1 test case... test.cpp(12): error: in "test1": check x == y has failed [10 != 10.000000099999999] test.cpp(14): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005] *** 2 failures are detected in the test module "tolerance_02"

Output

> tolerance_02
Running 1 test case...
test.cpp(12): error: in "test1": check x == y has failed [10 != 10.000000099999999]
test.cpp(14): error: in "test1": check x == z has failed [10 != 10.000999999999999]. Relative difference exceeds tolerance [0.0001 > 1e-005]

*** 2 failures are detected in the test module "tolerance_02"

	Caution
	The support for manipulators requires that your compiler supports variadic macros, `auto` for type deduction and `decltype`. These are C++11 features, but are also available on some pre-C++11 compilers. On compilers that are lacking these features resort to defining tolerance per test unit, or to compatibility test assertions: `BOOST_CHECK_CLOSE` and `BOOST_CHECK_SMALL`.

It is possible to specify the tolerance as percentage. At test unit level, the decorator syntax is:

* boost::unit_test::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) )
// equivalent to: boost::unit_test::tolerance( 2.0 / 100 )

At assertion level, the manipulator syntax is:

2.0% boost::test_tools::tolerance()
boost::test_tools::tolerance( boost::test_tools::fpc::percent_tolerance(2.0) )
// both equivalent to: boost::test_tools::tolerance( 2.0 / 100 )

Manipulator tolerance specifies the tolerance only for a single floating-point type. This type is deduced from form the numeric value passed along the manipulator:

expression	semantics
`tolerance(0.5)`	tolerance for type `double` changed to 0.5
`tolerance(float(0.5))`	tolerance for type `float` changed to 0.5
`tolerance(0.5f)`	tolerance for type `float` changed to 0.5
`tolerance(0.5L)`	tolerance for type `long double` changed to 0.5
`tolerance(Decimal("0.5"))`	tolerance for a user-defined type `Decimal` changed to the supplied value
`5.0% tolerance()`	tolerance for type `double` changed to 0.05 (`5.0 / 100`)
`5.0f% tolerance()`	tolerance for type `float` changed to 0.05
`Decimal("5.0")% tolerance()`	tolerance for type `Decimal` changed to value `(Decimal("5.0") / 100)`

This is also the case for decorator tolerance. In case of the decorator, however, it is possible to apply multiple decorators tolerance defining the tolerance for different types.

When values of two different floating point types T and U are compared, BOOST_TEST uses the tolerance specified for type boost::common_type<T, U>::type. For instance, when setting a tolerance for mixed float-to-double comparison, the tolerance for type double needs to be set.

Given two floating point types T and U and their common type C, the tolerance specified for type C is applied only when types T and U are appear as sub-expressions of the full expression inside assertion BOOST_TEST. It is not applied when T and U are compared inside a function invoked during the evaluation of the expression:

BOOST_AUTO_TEST_CASE(test, * utf::tolerance(0.02))
{
  double                  d1 = 1.00, d2 = 0.99;
  boost::optional<double> o1 = 1.00, o2 = 0.99;

  BOOST_TEST(d1 == d2);   // with tolerance    (double vs. double)
  BOOST_TEST(o1 == o2);   // without tolerance (optional vs. optional)
  BOOST_TEST(o1 == d2);   // without tolerance (optional vs. double)
  BOOST_TEST(*o1 == *o2); // with tolerance    (double vs. double)
}

Finally, note that comparisons for tolerance are also applied to operator< with semantics 'less by more than some tolerance`, and other relational operators. Also, the tolerance-based comparisons are involved when a more complicated expression tree is processed within the assertion body:

Example: tolerance applied in more complex expressions

Code
#define BOOST_TEST_MODULE tolerance_03 #include <boost/test/included/unit_test.hpp> namespace utf = boost::unit_test; double x = 10.000000; double d = 0.000001; BOOST_AUTO_TEST_CASE(passing, * utf::tolerance(0.0001)) { BOOST_TEST(x == x + d); // equal with tolerance BOOST_TEST(x >= x + d); // ==> greater-or-equal BOOST_TEST(d == .0); // small with tolerance } BOOST_AUTO_TEST_CASE(failing, * utf::tolerance(0.0001)) { BOOST_TEST(x - d < x); // less, but still too close BOOST_TEST(x - d != x); // unequal but too close BOOST_TEST(d > .0); // positive, but too small BOOST_TEST(d < .0); // not sufficiently negative }

Code

#define BOOST_TEST_MODULE tolerance_03
#include <boost/test/included/unit_test.hpp>
namespace utf = boost::unit_test;

double x = 10.000000;
double d =  0.000001;

BOOST_AUTO_TEST_CASE(passing, * utf::tolerance(0.0001))
{
  BOOST_TEST(x == x + d); // equal with tolerance
  BOOST_TEST(x >= x + d); // ==> greater-or-equal

  BOOST_TEST(d == .0);    // small with tolerance
}

BOOST_AUTO_TEST_CASE(failing, * utf::tolerance(0.0001))
{
  BOOST_TEST(x - d <  x); // less, but still too close
  BOOST_TEST(x - d != x); // unequal but too close

  BOOST_TEST(d > .0);     // positive, but too small
  BOOST_TEST(d < .0);     // not sufficiently negative
}

Output
> tolerance_03 Running 2 test cases... test.cpp(18): error: in "failing": check x - d < x has failed [10 - 1.0000000000000001e-05 >= 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(19): error: in "failing": check x - d != x has failed [10 - 1.0000000000000001e-05 == 10]. Relative difference is within tolerance [1e-06 < 0.001] test.cpp(21): error: in "failing": check d > .0 has failed [1.0000000000000001e-05 <= 0]. Absolute value is within tolerance [\|1e-05\| < 0.001] test.cpp(22): error: in "failing": check d < .0 has failed [1.0000000000000001e-05 >= 0] *** 4 failures are detected in the test module "tolerance_03"

Output

> tolerance_03
Running 2 test cases...
test.cpp(18): error: in "failing": check x - d < x has failed [10 - 1.0000000000000001e-05 >= 10]. Relative difference is within tolerance [1e-06 < 0.001]
test.cpp(19): error: in "failing": check x - d != x has failed [10 - 1.0000000000000001e-05 == 10]. Relative difference is within tolerance [1e-06 < 0.001]
test.cpp(21): error: in "failing": check d > .0 has failed [1.0000000000000001e-05 <= 0]. Absolute value is within tolerance [|1e-05| < 0.001]
test.cpp(22): error: in "failing": check d < .0 has failed [1.0000000000000001e-05 >= 0]

*** 4 failures are detected in the test module "tolerance_03"

Boost C++ Libraries

Floating point comparison

Example: specifying tolerance per test case

Example: specifying tolerance per assertion

Example: tolerance applied in more complex expressions