Boost C++ Libraries

In general, you should make sure that test cases are independent on one another. For any two test cases TA and TB, TB must not take for granted that TA has already executed, even if TA is declared before TB in the same translation unit.

The only ordering-related guarantee that Unit Test Framework makes by default is that if test cases TA and TB are declared in the same test suite, no test case (call it TX) from any other test suite is executed between TA and TB, even if the declaration of TX appears between the declarations of TA and TB. In other words, all tests from a suite are executed in one go, even if the test suite namespace is opened multiple times.

Even though the order is not guaranteed, it may be accidentally preserved across the different runs. In order to make sure the test cases do not depend on one another, whoever runs the test module may provide a command-line argument random to make sure the test units are shuffled within the suites.

Declaring a test case dependency

If an ordering dependency is required between the test units within the same suite, it has to be declared explicitly. Dependencies in the Unit Test Framework affect two dimensions of test units, which are:

Decorator depends_on associates the decorated test case (call it TB) with another test case (call it TA) specified by name. This affects the processing the test tree in two ways. First, test case TA is ordered to be run before TB, irrespective of the order in which they were declared or added to the test tree. Second, the execution of TB is skipped if TA is either disabled or skipped or is executed and marked as failed.

Example: decorator depends_on

#define BOOST_TEST_MODULE decorator_07
#include <boost/test/included/unit_test.hpp>

namespace utf = boost::unit_test;

// test1 and test2 defined at the bottom

BOOST_AUTO_TEST_CASE(test3, * utf::depends_on("s1/test1"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_CASE(test4, * utf::depends_on("test3"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_CASE(test5, * utf::depends_on("s1/test2"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_SUITE(s1)

  BOOST_AUTO_TEST_CASE(test1)
  {
    BOOST_TEST(true);
  }

  BOOST_AUTO_TEST_CASE(test2, * utf::disabled())
  {
    BOOST_TEST(false);
  }

BOOST_AUTO_TEST_SUITE_END()

> decorator_07 --report_level=detailed
Running 4 test cases...
test.cpp(10): error: in "test3": check false has failed

Test module "decorator_07" has failed with:
  1 test case out of 4 passed
  1 test case out of 4 failed
  2 test cases out of 4 skipped
  1 assertion out of 2 passed
  1 assertion out of 2 failed

  Test case "test3" has failed with:
    1 assertion out of 1 failed

  Test case "test4" was skipped
  Test case "test5" was skipped
  Test suite "s1" has passed with:
    1 test case out of 1 passed
    1 assertion out of 1 passed

    Test case "s1/test1" has passed with:
      1 assertion out of 1 passed

In the above scenario, test case test3 is run (and fails) because s1/test1 has been run and succeeded; test4 is skipped because test3 has failed; test5 is skipped because s1/test2 is disabled.