Tutorial: CodeCoverage aggregation using ctest

[rayment]

Tutorial

This was written against commit 1fcf7f4 in June 2024. The latest at the time of writing.

I am putting this here as a tutorial only. In my example, I use doctest bootstrapped by ctest but you can use Catch2 or something else instead. ctest will still be required though.

My project contains multiple shared libraries with separate tests. I want to be able to run lcov over the entire project and aggregate the results rather than deal with separate reports.

Some attempts were made previously by people in #8 with ctest and catkin. I have adapted and summarised some of this below.

Structure

My general project structure is as follows:

project_root/
    |- my_first_lib/
    |   |- include/
    |   |   |- ...
    |   |- src/
    |   |   |- ...
    |   |- test/
    |   |   |- ...
    |   |- CMakeLists.txt
    |- my_second_lib/
    |   |- include/
    |   |   |- ...
    |   |- src/
    |   |   |- ...
    |   |- test/
    |   |   |- ...
    |   |- CMakeLists.txt
    |- test/
    |   |- CMakeLists.txt
    |   |- main.cpp
    |- CMakeLists.txt

Whether or not you separate your include/ or src/ folders is irrelevant. What is relevant, is that you have an overarching test/ folder in the project root and separate test/ folders for each of your libraries/executables.

Code

project_root/CMakeLists.txt:

cmake_minimum_required(VERSION 3.27)

project(my_project LANGUAGES C CXX)
# bla bla bla, regular project root setup

# important parts below
enable_testing()

add_subdirectory(my_first_lib)
add_subdirectory(my_second_lib)

add_subdirectory(test)

project_root/my_first_lib/CMakeLists.txt:

# just your regular library setup below
add_library(my_first_lib SHARED)

target_sources(
    my_first_lib PRIVATE
    # include all of your library source files as you normally would
    src/library.cpp
)

target_include_directories(
    my_first_lib PUBLIC
    "${CMAKE_CURRENT_SOURCE_DIR}/include"
)

target_include_directories(
    my_first_lib PRIVATE
    "${CMAKE_CURRENT_SOURCE_DIR}/src"
)

# important part for testing
add_executable(my_first_lib_test)

target_sources(
    my_first_lib_test PRIVATE
    # I use an identical main() file for all tests, you can customise as required
    "${CMAKE_SOURCE_DIR}/test/main.cpp"
    # now put each of the test sources below
    test/my_first_lib_test_foobar.cpp
    test/my_first_lib_test_simple.cpp
    test/my_first_lib_test_very_hard.cpp
)


target_link_libraries(
    my_first_lib_test PRIVATE
    # I use doctest for testing, you may use something else
    doctest::doctest
    # the test executable links to the library
    my_first_lib
)

target_include_directories(
    my_first_lib_test PRIVATE
    "${CMAKE_SOURCE_DIR}/test"
)

add_test(NAME my_first_lib_ctest COMMAND my_first_lib_test)

append_coverage_compiler_flags_to_target(my_first_lib)
append_coverage_compiler_flags_to_target(my_first_lib_test)

project_root/my_second_lib/CMakeLists.txt:

Exactly the same as project_root/my_first_lib/CMakeLists.txt except everything targets my_second_lib_xxx rather than my_first_lib_xxx.

project_root/test/CMakeLists.txt:

setup_target_for_coverage_lcov(
    NAME "my_coverage"
    EXECUTABLE "${CMAKE_CTEST_COMMAND}" "--verbose"
    DEPENDENCIES
    # place all add_test targets below
    "my_first_lib_test"
    "my_second_lib_test"
    # exclusions must contain "test" to avoid including the main.cpp file
    # (your mileage may vary if you write your main() functions differently)
    EXCLUDE "/usr/*" "*/_deps/*" "test"
)

project_root/test/main.cpp:

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include <doctest/doctest.h>

Result

This results in a target my_coverage which will execute ctest which will execute all of your separate test executables.

I am using doctest and my tests will call into library code. In the same vein, you can adapt this to run an executable instead of a library by using a custom main() that calls all doctest test-cases if CMake is building in non-release mode.

Once lcov runs and generates the final report, all aggregated results will be listed from both libraries.

Further adaptation

If you don't use doctest or you don't care for a shared main() function, then you can simplify this by removing the project_root/test/ folder entirely and moving the setup_target_for_coverage_lcov call straight into project_root/CMakeLists.txt.