Build & run C++ unit tests with CMake

Welcome to the next pikoTutorial!

Building and running a single test file

To present the easiest case, we need to assume some folder structure of our project:

project
|- build
|- src
|  |- CMakeLists.txt
|  |- lib.cpp
|  |- lib.hpp
|- test
|  |- CMakeLists.txt
|  |- test_lib.cpp
|- thirdparty
|  |- googletest
|- CMakeLists.txt

Here the easiest case means that we have only on library and only one test file for it. Firstly of all, we need to put enable_testing() to our top level CMakeLists.txt file:

cmake_minimum_required(VERSION 3.15)
project(UnitTestsCMake)
# this line initializes testing environment and generate test target
enable_testing()

add_subdirectory(src)
add_subdirectory(test)
add_subdirectory(thirdparty/googletest)

CMakeLists.txt in src folder is just a one liner defining the library which we can link against in the tests:

add_library(MyLib lib.cpp)

Now the most important one – CMakeLists.txt file in test folder:

# create test target name
set(TARGET TestLib)
# create test executable
add_executable(${TARGET} test_lib.cpp)
# link our testd lib MyLib to the test executable declared above
target_link_libraries(${TARGET} PRIVATE gtest_main MyLib)
# define a test with its name and command to run it - in our simple case,
# test name is the same as command to run the test (the executable), but in
# command you can pass more information, e.g. the command line arguments
add_test(NAME ${TARGET} COMMAND ${TARGET})

After that the only thing left to do is to go to terminal and configure, build and run the unit tests:

cd build
cmake ..
cmake --build .
ctest

If everything went ok, you should see the following output:

Test project /home/pikotutorial/unit_tests_cmake/build
    Start 1: TestLib
1/1 Test #1: TestLib ..........................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.00 sec

You can always make this output more verbose by calling ctest with -V option:

UpdateCTestConfiguration  from :/home/pikotutorial/unit_tests_cmake/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/pikotutorial/unit_tests_cmake/build/DartConfiguration.tcl
Test project /home/pikotutorial/unit_tests_cmake/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: TestLib

1: Test command: /home/pikotutorial/unit_tests_cmake/build/test/TestLib
1: Working Directory: /home/pikotutorial/unit_tests_cmake/build/test
1: Test timeout computed to be: 10000000
1: Running main() from /home/pikotutorial/unit_tests_cmake/thirdparty/googletest/googletest/src/gtest_main.cc
1: [==========] Running 0 tests from 0 test suites.
1: [==========] 0 tests from 0 test suites ran. (0 ms total)
1: [  PASSED  ] 0 tests.
1/1 Test #1: TestLib ..........................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.01 sec

Creating a test suite

After learning how to define a single executable with unit tests, we can move to slightly more complex example in which there are multiple test files. Look at this project structure:

project
|- build
|- src
  |- CMakeLists.txt
  |- lib.cpp
  |- lib.hpp
|- test
  |- CMakeLists.txt
  |- test_lib_1.cpp
  |- test_lib_2.cpp
  |- test_lib_3.cpp
|- thirdparty
  |- googletest
|-CMakeLists.txt

You could of course repeat 3 times what we did above, but because every test requires 4 lines, as the number of tests increases, your CMakeLists.txt file would quickly start to be very long hard to maintain. This is the reason why it’s much better to do this using loop:

# define a list of test names
set(LIB_TESTS
    test_lib_1
    test_lib_2
    test_lib_3
)
# iterate over every test
foreach(TEST_NAME ${LIB_TESTS})
    add_executable(${TEST_NAME} ${TEST_NAME}.cpp)
    target_link_libraries(${TEST_NAME} PRIVATE gtest_main MyLib)
    add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME})
endforeach()

This way, whenever you add a new test file, you just need to append its name to the LIB_TESTS list and that’s it. After that you can run all your tests with a single command ctest what should give you the following result.

Test project /home/pawbar/Documents/projekty/piko_tutorial/content/sop_24_07/cpp/unit_tests_cmake/build
    Start 1: test_lib_1
1/3 Test #1: test_lib_1 .......................   Passed    0.01 sec
    Start 2: test_lib_2
2/3 Test #2: test_lib_2 .......................   Passed    0.01 sec
    Start 3: test_lib_3
3/3 Test #3: test_lib_3 .......................   Passed    0.01 sec

100% tests passed, 0 tests failed out of 3

Total Test time (real) =   0.03 sec

Creating multiple test suites

Now let’s jump into the structure which is the most commonly met in various software projects. Previous examples provided overview on how to use CMake for unit testing, but in real projects you would very rarely meet structures with only one file with tests or only one test suite which. Usually, there are multiple test files organized in multiple test suites. Moreover, during a typical work you want to run unit tests only from the test suite that you’re currently working on, not all unit tests every time. Let’s then mimic such structure trying at the same time to keep it simple:

project
|- build
|- src
  |- CMakeLists.txt
  |- lib_1.cpp
  |- lib_1.hpp
  |- lib_2.cpp
  |- lib_2.hpp
|- test
  |- CMakeLists.txt
  |- test_lib_1_1.cpp
  |- test_lib_1_2.cpp
  |- test_lib_2_1.cpp
  |- test_lib_2_2.cpp
|- thirdparty
  |- googletest
|-CMakeLists.txt

Again, you could just duplicate the code for creating a single test suite from the previous example, but there are 2 problems with that:

how to differentiate between the test suites, so that tests from only one can be ran?
how to avoid code duplication related to repeated loops?

First problem can be solved with labels – each test may have a label assigned and all tests with the same labels create a single test suite:

set(LIB_1_TESTS
    test_lib_1_1
    test_lib_1_2
)

foreach(TEST_NAME ${LIB_1_TESTS})
    add_executable(${TEST_NAME} ${TEST_NAME}.cpp)
    target_link_libraries(${TEST_NAME} PRIVATE gtest_main MyLib1)
    add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME})
    set_tests_properties(${TEST_NAME} PROPERTIES LABELS "test_lib_1")
endforeach()

set(LIB_2_TESTS
    test_lib_2_1
    test_lib_2_2
)

foreach(TEST_NAME ${LIB_2_TESTS})
    add_executable(${TEST_NAME} ${TEST_NAME}.cpp)
    target_link_libraries(${TEST_NAME} PRIVATE gtest_main MyLib2)
    add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME})
    set_tests_properties(${TEST_NAME} PROPERTIES LABELS "test_lib_2")
endforeach()

After setting labels, to run e.g. only test suite test_lib_1, call:

ctest -L test_lib_1

What will run tests from only 2 files, instead of all of them:

Test project /home/pikotutorial/unit_tests_cmake/build
    Start 1: test_lib_1_1
1/2 Test #1: test_lib_1_1 .....................   Passed    0.00 sec
    Start 2: test_lib_1_2
2/2 Test #2: test_lib_1_2 .....................   Passed    0.00 sec

100% tests passed, 0 tests failed out of 2

Label Time Summary:
test_lib_1    =   0.00 sec*proc (2 tests)

Total Test time (real) =   0.00 sec

To avoid the code duplication, we will create a CMake function:

# define a function creating a single test suite
function(create_test_suite TEST_SUITE_NAME TESTS DEPENDENCIES)
    foreach(TEST_NAME IN LISTS TESTS)
        add_executable(${TEST_NAME} ${TEST_NAME}.cpp)
        target_link_libraries(${TEST_NAME} PRIVATE gtest_main ${DEPENDENCIES})
        add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME})
        set_tests_properties(${TEST_NAME} PROPERTIES LABELS ${TEST_SUITE_NAME})
    endforeach()
endfunction()
# define a list of tests in the first test suite
set(LIB_1_TESTS
    test_lib_1_1
    test_lib_1_2
)
# define a list of tests in the second test suite
set(LIB_2_TESTS
    test_lib_2_1
    test_lib_2_2
)
# use create_test_suite function to define 2 test suites
create_test_suite("test_lib_1" "${LIB_1_TESTS}" MyLib1)
create_test_suite("test_lib_2" "${LIB_2_TESTS}" MyLib2)

Note for beginners: I didn’t want to overcomplicate the folder structure here, but in reality usually you want to organize your test suite into separate test_* folders coresponding to the folders where the source code is located. In such case, every folder should have a dedicated CMakeLists.txt files which defines a test suite for the given folder.

Note for advanced: in big projects built under a single top-level CMake with sub-components owned by a different teams, there’s always a discussion about the strategy on avoiding labels duplication. One of them is to simply use paths relative to the project root. For example, a test/test_lib_1/CMakeLists.txt file would define a test suite labeled as test/test_lib_1. Thanks to such approach, when someone wants to run all the unit tests from this folder, the command is ctest -L test/test_lib_1. This is not only clear, but also allows to utilize the Tab autocompletion in the terminal when choosing the test suite because the label corresponds to the folder structure.

Building and running at once

So far so good, but there’s still one thing missing for the full convenience. Note that the only thing that ctest does is running the test binaries that have already been built. Whenever you introduce a change to your code and want to check if tests are still passing, you need to first compile your tests and then run the tests. It would be nice of course to be able to do that in a single command. To achieve that we can add a custom target to the create_test_suite function:

function(create_test_suite TEST_SUITE_NAME TESTS DEPENDENCIES)
    foreach(TEST_NAME IN LISTS TESTS)
        add_executable(${TEST_NAME} ${TEST_NAME}.cpp)
        target_link_libraries(${TEST_NAME} PRIVATE gtest_main ${DEPENDENCIES})
        add_test(NAME ${TEST_NAME} COMMAND ${TEST_NAME})
        set_tests_properties(${TEST_NAME} PROPERTIES LABELS ${TEST_SUITE_NAME})
    endforeach()
    # add a target which runs tests for the given label and depends on the given tests
    add_custom_target(${TEST_SUITE_NAME}
        COMMAND ${CMAKE_CTEST_COMMAND} -L ${TEST_SUITE_NAME}
        DEPENDS ${TESTS}
    )
endfunction()

Since now, after every code change, if you want to recompile and run test_lib_1 test suite, you can simply use command:

cmake --build . --target test_lib_1