froglogic / Blog / Tip of the Week / Unit Tests for Qt-based Applications with Catch

Unit Tests for Qt-based Applications with Catch

Unit tests for Qt-based applications with Catch

Squish for Qt is a perfect fit for testing Qt based user interfaces, however an application always consists of quite some backend code as well. Unit tests are one central piece of testing this backend code.

There are numerous unit testing frameworks and libraries available for C++.
For Qt-based code the most natural choice is the QTest library that ships with Qt. But there are cases where using QTest may not be the best choice:

  • Application code that supports multiple Qt versions with different feature sets of QTest (or no QTest at all in case of Qt 3)
  • QTest wants you to create QObject subclasses and slots to model your tests which adds some boilerplate code and limits how tests can be structured
  • QTest has a rather simple report format that does not allow for structured reporting, this needs to be countered by splitting up tests into smaller tests
  • Projects with both Qt-based and non-Qt code may need a unit testing framework without a Qt dependency for the non-Qt parts

A possible alternative to QTest is Catch, a single-include test framework for writing unit tests in C++. Unlike QTest it does not depend on Qt. In fact it has no external dependencies at all. Structuring test cases and tests in Catch is remarkably concise since it uses macros and standard C++ blocks. Apart from the root of a test being wrapped into TEST_CASE() { }, Catch enforces no further structure, however it does support structuring. Additionally it supports BDD-like structuring of tests so you can group your tests into Given-, When- and Then-blocks which fit very well into the notion of pre- and post-conditions. Finally Catch 1.x has very modest C++ standard requirements and works on many C++98 capable compilers. But of course also for Catch there are reasons why it may not be the best choice either:

  • Compiling the Catch main() function takes considerable time (can be countered by putting main() into own source file)
  • Handling of Qt specific types and concepts like the Qt event-loop and signals are not supported out of the box
  • Catch does not execute tests in an isolated manner, i.e. in separate processes (but then QTest does not either)

The basics on how to use Catch are documented in a tutorial. This article will focus on how testing code that uses Qt works inside a Catch unit test and will use Catch 1.x to support older platforms and compilers that lack the C++11 support needed by Catch 2.x. Most of the following should also apply to and work with Catch 2.x though.

Testing Qt code with Catch

Let’s start with an artificial example of how to write tests with Catch that also involve Qt types. We’ll start with the main() function which is provided by Catch itself:

// main.cpp
#include "catch/catch.hpp"

The above goes into a file of its own to avoid slow compiles when adding more tests later on.

We also need some tests to show how things work and look like:

// some_tests.cpp
#include "catch/catch.hpp"
#include <QString>

TEST_CASE("Some.Tests", "")
    SECTION("Something", "")
        CHECK(QString() == QString());
        const QString someString(QLatin1String("foo"));
        CHECK(someString == QLatin1String("bar"));

And finally a small qmake project to build the test application:

QT = core
SOURCES = main.cpp some_tests.cpp

When executing the resulting test application, the second CHECK() will produce a failure (as expected). However the output is not quite as expected yet:

catch_qt is a Catch v1.12.1 host application.
Run with -? for options


some_tests.cpp:13: FAILED:
  CHECK( someString == QLatin1String("bar") )
with expansion:
  {?} == {?}

test cases: 1 | 1 failed
assertions: 3 | 2 passed | 1 failed

In the above output the expansion of the compared values did not work and Catch instead just shows {?} for the left and right side of the comparison. This is of course not very helpful, one needs to see the actual value of a variable for such a failing test.

Improving Qt type output

Fixing the textual output for value expansions is possible by adding streaming operators for
involved types as explained in the Catch documentation about string conversions.
For Qt types adding std::ostream operators can look like this:

inline std::ostream &operator<<(std::ostream &os, const QByteArray &value)
    return os << '"' << (value.isEmpty() ? "" : value.constData()) << '"';

inline std::ostream &operator<<(std::ostream &os, const QLatin1String &value)
    return os << '"' << value.latin1() << '"';

inline std::ostream &operator<<(std::ostream &os, const QString &value)
    return os << value.toLocal8Bit();

The result already looks more helpful:

some_tests.cpp:13: FAILED:
  CHECK( someString == QLatin1String("bar") )
with expansion:
  "foo" == "bar"

Testing Qt code that needs an application object

Some function and classes in Qt need a Qt application object to do their work correctly. One way to achieve that would be by adding it at the TEST_CASE scope:

TEST_CASE("Some.Tests", "")
    QCoreApplication app;
    SECTION("Test this", "")
        // some tests
    SECTION("Test that", "")
        // more tests here

There is however a drawback of doing it like that. Since every execution of a SECTION will also execute the surrounding code, the test would create and destroy the app instance for each section. This behavior is great for isolating tests from each other, in case of the central Qt application object however creating it exactly once is probably the better solution.

Supporting this is possible by providing a custom main() function that creates a QCoreApplication. The basics for doing this are covered in the Catch documentation,
an implementation could look like this:

// main.cpp
#include <QCoreApplication>
#include "catch/catch.hpp" // include after defining CATCH_CONFIG_RUNNER
int main(int argc, char **argv)
    QCoreApplication app(argc, argv);
    const int res = Catch::Session().run(argc, argv);
    return (res < 0xff ? res : 0xff);

In addition to creating the application object only once it also allows for Qt to parse its
own commandline arguments like -platform which is sometimes used to run parts of a UI application in a headless environment.

Final thoughts

All of the above can be combined into a single header file, just like Catch itself. With some preprocessor magic around it will even work similar to using the catch.cpp header. A complete example test including all of the above code is available for download as

There is of course always room for improvement. It would probably be helpful to extend the Qt integration to support watching Qt signals or for verifying classes that need a running eventloop to do their work. The QTest library is catering for such needs but Qt itself offers enough to achieve similar results using any other C++ unit test framework.

Leave a Reply

Your email address will not be published. Required fields are marked *

Copy link
Powered by Social Snap