Debugging Test Failures Using Your Favorite IDE

Debugging Test Failures Using Your Favorite IDE

One goal of automated testing through Squish is to find bugs and regressions in your application. The cause for some test failures is easy to spot, other ones can be hard to debug. Squish already provides crash dumps when the AUT decides to give up, but in not-so-obvious cases you might want to take a look with a debugger. One way to do that is to attach the debugger to the running application process. When debugging an issue it can however be more convenient to run your application in debug mode right from your IDE. This way you can experiment with it using Squish while observing behavior from the inside.

As you might know, Squish is able to attach to applications that are already running instead of starting them as part of the test script. The feature is useful to test AUTs running on embedded devices for example, but we can use it here just as well. The idea is to register an “attachable AUT” with Squish and to add a run configuration to your IDE that starts the AUT in “attachable mode”. Let’s first set it up in Squish and then look at specific IDE’s to see how it’s done.

Registering The Attachable AUT With Squish

From the Squish IDE, select “Edit→Server Settings→Manage AUTs…”, click “Attachable AUTs” and then choose “Add…”. In the dialog that opens enter a name and an arbitrary (but free) port on the machine where the AUT will be running. The name will be what goes into the attachToApplication script call later.

Now Squish knows about the application, so let’s configure the IDE to launch it accordingly.

Qt Creator 4

Qt Creator supports the Custom Executable run configuration type that allows us to run a, well, custom executable when launching the application. Open the Projects view and select the Run subcategory of the Qt version you wish to use. Then, select “Add→Custom Executable”.

Now you need to configure the executable and arguments for it.

  • Executable: Choose the “startaut” (Qt applications) or “startwinaut” (non-Qt Windows applications) executable from $SQUISH_PREFIX/bin. This little helper hooks your application and opens a port for the remote control connection to Squish.
  • Command Line Arguments: These are the arguments that need to be passed to startaut. Set it to “–port=45632 %{CurrentRun:Executable:FilePath}” — where the port is the one you chose when registering the AUT!
    Caution:
    In Qt Creator 4.3 there’s a bug with the FilePath variable. Use the full path to the compiled AUT binary instead.

Qt Creator screenshot with the "Add" dropdown for adding a run configuration open and the "custom executable" entry highlighted.

Visual Studio 2015

The setup in Visual Studio is quite similar to the one in Qt Creator. To keep things nicely separated you might want to first create a new Configuration in the Configuration Manager opened via the Debug/Release-Dropdown.

Visual Studio screenshot with the Run Configuration dropdown open and the "Configuration Manager..." entry selected.

Make sure to copy the settings from the configuration you usually use to have everything in place:

Visual Studio's configuration manager dialog with the "New Solution Configuration" dialog in front of it and the "Copy Settings From" option highlighted.

Now open the project properties (Project→<Appname> Properties…) and switch to the Debug page. There you should see a Start Action menu as well as Start Options. These need to be set up almost just like above:

  • Start External Program: Choose the “startaut.exe” (Qt applications) or “startwinaut.exe” (non-Qt Windows applications) executable from $SQUISH_PREFIX/bin. This little helper hooks your application and opens a port for the remote control connection to Squish.
  • Command Line Arguments: These are the arguments that need to be passed to startwinaut. Set it to “–port=45632 C:\path\to\your\application.exe” — where the port is the one you chose when registering the AUT! (Apparently there’s no built-in way to use a variable here, so you’ll have to put the literal path to the executable in.)

Visual Studio's project settings with "start external program" and "command line arguments" input fields highlighted.

Almost Done

Now launch your AUT via the IDE. After it has started up, you can run your test scripts as usual – the only difference is that you need to replace calls to startApplication with attachToApplication. That’s it!

0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*