Benefiting from combined usage of Squish and Coco

Benefiting from combined usage of Squish and Coco

It is sometimes a good idea to use Squish GUI Tester and Coco in the same project. Squish runs the GUI tests, and Coco measures how much code they cover. A problem that often occurs is however that the instrumented AUT does not write any coverage results.

This is caused by the way Squish and Coco work. The default for Squish is to start the AUT at the beginning of a test and to stop it when the test ends. An AUT that is instrumented by Coco on the other hand writes the coverage data to the disk just before it finishes. A problem occurs when the AUT is killed by Squish while it is still writing coverage data to its .csexe file. This can especially occur if the program is big and has a lot of data to write.

Giving the AUT enough time to stop

The natural solution for this problem is to give the AUT enough time to write its data. There are two timeouts in the Squish Runner that are involved when a Squish test script finishes.

  • SoftExitTimeout. After the test script has ended, the Squish Server waits for this time until it sends the AUT a signal to close. The AUT then still runs and can do its own cleanup activity. (This timeout works only under Unix.)
  • HardExitTimeout. After the signal has been sent, the Squish Server waits for this time until it sends a “hard kill” signal to the AUT, which stops all of its activity immediately.

Both timeouts are measured in milliseconds. They are set with the --config option of squishrunner, like this:

$ squishrunner --config setHardExitTimeout 4000

This command sets the HardExitTimeout to 4 seconds.

Practical use

The procedure to follow is somewhat different between Unix and Windows systems.

Under Unix, when a test case ends and the AUT is still running, Squish waits until SoftExitTimeout is over. Then it sends a signal to close to the AUT, which  writes the coverage data to the .csmes file and finishes. Meanwhile, Squish waits for the time given by HardExitTimeout for the AUT to stop. If the AUT is still active, Squish will terminate it — and the .csmes file will not be written. Setting HardExitTimeout to a high enough value is therefore enough under Unix.

Under Windows, there is no SoftExitTimeout. It is therefore necessary that the test script closes the AUT explicitly. This can be done by clicking on a “Close” button, selecting the “File->Exit” menu entry, or anything else that is supported by the AUT. Then the AUT writes its .csexe file when the test case ends, and with a high enough HardExitTimeout it will also have time enough to write it.

A high enough value can most easily found out by experiment: If an AUT does not write a .csexe file in a test case, try a higher timeout and start the test case again.

Postscript

There is also a simple check that is sometimes useful: Run the instrumented AUT without Squish and see whether it then writes a .csexe file. If it does not, the problem is somewhere else.

0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*