...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
Instead of comparing a single value against another, there is often a need for comparing collections of values. A collection and indirectly the values it contains may be considered in several ways:
N
values are stored in a container. Containers in this case are used
for storing several values, and iterating over the containers yields
sequences that can be compared element-wise.
The iteration should be in an order that is a priori
known [10], for being able to compare the sequences. The values in
the collection are independent each other, and subsets can be compared
as well.
N
representing a point in a N
dimensional space, compared to another point with the relation "<
": the comparison is application
specific and a possible comparison would be the lexicographical ordering
[11].
The following observations can be done:
BOOST_TEST
provides specific tools
for comparing collections:
More details about the concept of collection in the Unit Test Framework is given here.
The default comparison dispatches to the existing overloaded comparison operator. The Unit Test Framework distinguishes two use cases
std::vector
behaviour
Given two containers c_a
and c_b
that are not C-arrays,
BOOST_TEST(c_a op c_b)
is equivalent, in terms of test success, to
auto result = c_a op c_b; BOOST_TEST(result);
In the example below, operator==
is not defined for std::vector
of different types, and the program would fail to compile if the corresponding
lines were uncommented (std::vector
uses lexicographical comparison by default).
Note | |
---|---|
In the case of default comparison, there is no additional diagnostic
provided by the Unit Test Framework. See the section
|
Code |
---|
#define BOOST_TEST_MODULE boost_test_sequence #include <boost/test/included/unit_test.hpp> #include <vector> BOOST_AUTO_TEST_CASE( test_collections_vectors ) { std::vector<int> a{1,2,3}, c{1,5,3,4}; std::vector<long> b{1,5,3}; // the following does not compile //BOOST_TEST(a == b); //BOOST_TEST(a <= b); // stl defaults to lexicographical comparison BOOST_TEST(a < c); BOOST_TEST(a >= c); BOOST_TEST(a != c); } |
Output |
---|
> ./boost_test_container_default --log_level=all Running 1 test case... Entering test module "boost_test_sequence" test.cpp(13): Entering test case "test_collections_vectors" test.cpp(23): info: check a < c has passed test.cpp(24): error: in "test_collections_vectors": check a >= c has failed test.cpp(25): info: check a != c has passed test.cpp(13): Leaving test case "test_collections_vectors"; testing time: 208us Leaving test module "boost_test_sequence"; testing time: 286us *** 1 failure is detected in the test module "boost_test_container_default" |
As soon as one of the operands is a C-array, there is no default
behaviour the Unit Test Framework can dispatch
to. This is why in that case, the comparison mimics the std::vector
behaviour.
Code |
---|
#define BOOST_TEST_MODULE boost_test_containers_c_arrays #include <boost/test/included/unit_test.hpp> #include <vector> namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE( test_collections_on_c_arrays ) { int a[] = {1, 2, 3}; int b[] = {1, 5, 3, 4}; std::vector<long> c{1, 5, 3, 4}; BOOST_TEST(a == a); // element-wise compare BOOST_TEST(a == b); // element-wise compare BOOST_TEST(a != b); BOOST_TEST(a < b); // lexicographical compare BOOST_TEST(b < c); // lexicographical compare BOOST_TEST(c < a); // lexicographical compare } |
Output |
---|
> ./boost_test_containers_c_arrays --log_level=all Running 1 test case... Entering test module "boost_test_containers_c_arrays" test.cpp:15: Entering test case "test_collections_on_c_arrays" test.cpp:20: info: check a == a has passed test.cpp:21: error: in "test_collections_on_c_arrays": check a == b has failed. Collections size mismatch: 3 != 4 test.cpp:22: info: check a != b has passed test.cpp:23: info: check a < b has passed test.cpp:24: error: in "test_collections_on_c_arrays": check b < c has failed. Collections appear to be equal. test.cpp:25: error: in "test_collections_on_c_arrays": check c < a has failed. Failure at position 1: 5 >= 2. test.cpp:15: Leaving test case "test_collections_on_c_arrays"; testing time: 204us Leaving test module "boost_test_containers_c_arrays"; testing time: 240us *** 3 failures are detected in the test module "boost_test_containers_c_arrays" |
By specifying the manipulator boost::test_tools::per_element
,
the comparison of the elements of the containers are performed element-wise,
in the order given by the forward iterators of the containers. This is
a comparison on the sequences of elements generated
by the containers, for which the Unit Test Framework
provides advanced diagnostic.
In more details, let c_a = (a_1,... a_n)
and c_b
= (b_1,... b_n)
be two sequences of same length, but not necessarily of same type. Those
sequences correspond to the content of the respective containers, in the
order given by their iterator. Let op
be one of the binary comparison
operators.
BOOST_TEST(c_a op c_b, boost::test_tools::per_element() );
is equivalent to
if(c_a.size() == c_b.size())
{
for(int i=0; i < c_a.size(); i++)
{
BOOST_TEST_CONTEXT
("index " << i)
{
BOOST_TEST(a_i op b_i);
}
}
}
else
{
BOOST_TEST(c_a.size() == c_b.size());
}
Warning | |
---|---|
this is fundamentally different from using the containers' default comparison operators (default behavior). |
Warning | |
---|---|
this is not an order relationship on containers. As a side effect, it is possible to have BOOST_TEST(c_a == c_b) and BOOST_TEST(c_a != c_b) failing at the same time |
Sequences are compared using the specified operator op
,
evaluated on the left and right elements of the respective sequences. The
order of the compared elements is given by the iterators of the respective
containers [13]. In case of failure, the indices of the elements failing op
are returned.
Code |
---|
#define BOOST_TEST_MODULE boost_test_sequence_per_element #include <boost/test/included/unit_test.hpp> #include <vector> #include <list> namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE( test_sequence_per_element ) { std::vector<int> a{1,2,3}; std::vector<long> b{1,5,3}; std::list<short> c{1,5,3,4}; BOOST_TEST(a == b, tt::per_element()); // nok: a[1] != b[1] BOOST_TEST(a != b, tt::per_element()); // nok: a[0] == b[0] ... BOOST_TEST(a <= b, tt::per_element()); // ok BOOST_TEST(b < c, tt::per_element()); // nok: size mismatch BOOST_TEST(b >= c, tt::per_element()); // nok: size mismatch BOOST_TEST(b != c, tt::per_element()); // nok: size mismatch } BOOST_AUTO_TEST_CASE( test_compare_c_arrays_element_wise ) { int a[] = {1, 2, 3}; int b[] = {1, 5, 3}; std::vector<long> c{1, 5, 3}; BOOST_TEST(a == b, boost::test_tools::per_element()); BOOST_TEST(a != b, boost::test_tools::per_element()); BOOST_TEST(a < b, boost::test_tools::per_element()); BOOST_TEST(b < c, boost::test_tools::per_element()); BOOST_TEST(c < a, boost::test_tools::per_element()); } |
Output |
---|
> ./boost_test_sequence_per_element Running 2 test cases... test.cpp:21: error: in "test_sequence_per_element": check a == b has failed Mismatch at position 1: 2 != 5. test.cpp:23: error: in "test_sequence_per_element": check a != b has failed Mismatch at position 0: 1 == 1. Mismatch at position 2: 3 == 3. test.cpp:25: error: in "test_sequence_per_element": check b < c has failed Collections size mismatch: 3 != 4 test.cpp:26: error: in "test_sequence_per_element": check b >= c has failed Collections size mismatch: 3 != 4 test.cpp:27: error: in "test_sequence_per_element": check b != c has failed Collections size mismatch: 3 != 4 test.cpp:35: error: in "test_compare_c_arrays_element_wise": check a == b has failed Mismatch at position 1: 2 != 5. test.cpp:36: error: in "test_compare_c_arrays_element_wise": check a != b has failed Mismatch at position 0: 1 == 1. Mismatch at position 2: 3 == 3. test.cpp:37: error: in "test_compare_c_arrays_element_wise": check a < b has failed Mismatch at position 0: 1 >= 1. Mismatch at position 2: 3 >= 3. test.cpp:38: error: in "test_compare_c_arrays_element_wise": check b < c has failed Mismatch at position 0: 1 >= 1. Mismatch at position 1: 5 >= 5. Mismatch at position 2: 3 >= 3. test.cpp:39: error: in "test_compare_c_arrays_element_wise": check c < a has failed Mismatch at position 0: 1 >= 1. Mismatch at position 1: 5 >= 2. Mismatch at position 2: 3 >= 3. *** 10 failures are detected in the test module "boost_test_sequence_per_element" |
For the sequences to be comparable element-wise, the following conditions should be met:
op
should be one of
the comparison operator ==
,
!=
, <
,
<=
, >
,
>=
a_i op
b_i
should be defined, where
the type of a_i
and
b_i
are the type returned
by the dereference operator of the respective collections.
Caution | |
---|---|
the resulting type of " BOOST_TEST(c_a == c_b == 42, boost::test_tools::per_element() ); // does not compile |
By specifying the manipulator boost::test_tools::lexicographic
,
the containers are compared using the lexicographical
order and for which the Unit Test Framework provides
additional diagnostic in case of failure.
BOOST_TEST(c_a op c_b, boost::test_tools::lexicographic() );
The comparison is performed in the order given by forward iterators of the containers.
Tip | |
---|---|
lexicographic comparison yields a total order on the containers: the
statements |
Note | |
---|---|
The equality |
Code |
---|
#define BOOST_TEST_MODULE boost_test_container_lex #include <boost/test/included/unit_test.hpp> #include <vector> namespace tt = boost::test_tools; BOOST_AUTO_TEST_CASE( test_collections_vectors_lex ) { std::vector<int> a{1,2,3}, b{1,2,2}, c{1,2,3,4}; BOOST_TEST(a < a, tt::lexicographic()); BOOST_TEST(a < b, tt::lexicographic()); BOOST_TEST(a < c, tt::lexicographic()); BOOST_TEST(a >= c, tt::lexicographic()); //BOOST_TEST(a == c, tt::lexicographic()); // does not compile //BOOST_TEST(a != c, tt::lexicographic()); // does not compile } BOOST_AUTO_TEST_CASE( test_compare_c_arrays_lexicographic ) { int a[] = {1, 2, 3}; int b[] = {1, 5, 3}; std::vector<long> c{1, 5, 3}; // BOOST_TEST(a == b, boost::test_tools::lexicographic()); // does not compile // BOOST_TEST(a != b, boost::test_tools::lexicographic()); // does not compile BOOST_TEST(a < b, boost::test_tools::lexicographic()); BOOST_TEST(b < c, boost::test_tools::lexicographic()); BOOST_TEST(c < a, boost::test_tools::lexicographic()); } |
Output |
---|
> ./boost_test_container_lex --log_level=all Running 2 test cases... Entering test module "boost_test_container_lex" test.cpp:15: Entering test case "test_collections_vectors_lex" test.cpp:19: error: in "test_collections_vectors_lex": check a < a has failed Collections appear to be equal. test.cpp:20: error: in "test_collections_vectors_lex": check a < b has failed Failure at position 2: 3 >= 2. test.cpp:21: info: check a < c has passed test.cpp:22: error: in "test_collections_vectors_lex": check a >= c has failed Second collection has extra trailing elements. test.cpp:15: Leaving test case "test_collections_vectors_lex"; testing time: 178us test.cpp:28: Entering test case "test_compare_c_arrays_lexicographic" test.cpp:35: info: check a < b has passed test.cpp:36: error: in "test_compare_c_arrays_lexicographic": check b < c has failed Collections appear to be equal. test.cpp:37: error: in "test_compare_c_arrays_lexicographic": check c < a has failed Failure at position 1: 5 >= 2. test.cpp:28: Leaving test case "test_compare_c_arrays_lexicographic"; testing time: 88us Leaving test module "boost_test_container_lex"; testing time: 323us *** 5 failures are detected in the test module "boost_test_container_lex" |
As seen above,
==
relation is either explicit (using boost::test_tools::per_element()
) or implicit when the container overloads/implements
this type of comparison,
<
(and derived operations) is either
explicit (using boost::test_tools::lexicographic()
) or implicit when the container overloads/implements
uses this type of comparison.
When the default is to using the container implementation, it is not possible
to benefit from an extended failure diagnostic. The Unit Test
Framework provides a mechanism for performing the same comparisons
through the Unit Test Framework instead of the container
operator, through the macro BOOST_TEST_SPECIALIZED_COLLECTION_COMPARE
that might be used as follow:
std::vector<int>
to lexicographic with extended diagnostic
Code |
---|
#define BOOST_TEST_MODULE boost_test_container_lex_default #include <boost/test/included/unit_test.hpp> #include <vector> namespace tt = boost::test_tools; BOOST_TEST_SPECIALIZED_COLLECTION_COMPARE(std::vector<int>) BOOST_AUTO_TEST_CASE( test_collections_vectors_lex ) { std::vector<int> a{1,2,3}, b{1,2,2}; std::vector<long int> c{1,2,3,5}, d{1,2,3,4}; BOOST_TEST(a < a); // lexcographic BOOST_TEST(a < b); // lexcographic BOOST_TEST(a == b); // element-wise BOOST_TEST(a != b); // extended diagnostic BOOST_TEST(c < d); // no extended diagnostic BOOST_TEST(c == d); // no extended diagnostic } |
Output |
---|
> ./boost_test_container_lex_default --log_level=all Running 1 test case... Entering test module "boost_test_container_lex_default" test.cpp:17: Entering test case "test_collections_vectors_lex" test.cpp:22: error: in "test_collections_vectors_lex": check a < a has failed. Collections appear to be equal. test.cpp:23: error: in "test_collections_vectors_lex": check a < b has failed. Failure at position 2: 3 >= 2. test.cpp:24: error: in "test_collections_vectors_lex": check a == b has failed. Mismatch at position 2: 3 != 2. test.cpp:25: info: check a != b has passed test.cpp:26: error: in "test_collections_vectors_lex": check c < d has failed test.cpp:27: error: in "test_collections_vectors_lex": check c == d has failed test.cpp:17: Leaving test case "test_collections_vectors_lex"; testing time: 155us Leaving test module "boost_test_container_lex_default"; testing time: 177us *** 5 failures are detected in the test module "boost_test_container_lex_default" |
op
should be one of
the comparison operator ==
,
!=
, <
,
<=
, >
,
>=
Note | |
---|---|
Note that the operation |
A sequence is given by the iteration over a forward iterable container. A forward iterable container is:
class
/struct
that implements the member functions
begin
and end
.
For collection comparisons, both sequences are also required to be different
than string
sequences.
In that case, the sequences are dispatched to string comparison
instead.
Warning | |
---|---|
|
Tip | |
---|---|
If the behaviour of |
Note | |
---|---|
Since Boost.Test 3.6 (Boost 1.65) the requirements for the collection concepts have been relaxed to include C-arrays as well |
Note | |
---|---|
Since Boost.Test 3.7 (Boost
1.67) the definition of |
The detection of the types that meet these requirements containers is delegated
to the class boost::unit_test::is_forward_iterable
,
which for C++11 detects the required member functions and fields. However
for C++03, the types providing the sequences should be explicitly indicated
to the Unit Test Framework by a specialization of
boost::unit_test::is_forward_iterable
[14].
[10]
this might not be the case for e.g. std::unordered_map
,
for which the buckets might be filled differently depending on the
insertion order.
[11]
in this case v_a < v_b
means that the point v_a
is inside the rectangle (origin, v_b
)
[12] either defined by the container or by the user
[13] the containers should yield the same sequences for a fixed set of elements they contain
[14]
Standard containers of the STL
are recognized as forward iterable container.