Verification points are one of the central aspects of automated testing. One part is to automatically drive the AUT. The other part is to verify that the application reacted as expected. This is done using verification points which verify that property values, the screen output or similar is as expected.
In Squish it is possible to insert verification points via a point & click interface in the Squish IDE, or to code them manually via script statements. Bot approaches are presented in this chapter.
The most common type of verifications is to compare the value of an object's property with an expected value. For example, after a text was entered, one could compare that the line editor's text property returns the expected value.
Such verification points can easily be inserted into a test script using Squish IDE's point & click interface. A detailed step-by-step description on how to insert an object property verification point can be found in the chapter Inserting a Verification Point (Section 5.2.1) in the tutorial Tutorial: Creating the First Test with Squish for Qt (Chapter 5).
Object property comparisons can be inserted for all properties of QObject based objects. This includes all widgets of the Qt toolkit and custom widgets based on QWidget.
Squish provides also extensions so items of view classes (such as QListView or Qt 4's item views) and menus can be accessed like any other QObject's property.
This means a QListView or QPopupMenu, for example, have an
item_
child for each list view or menu item. This item in return has
properties such as the item's text and the item's state (enabled,
disabled, checked, etc.). Items can have child items again (such as
nested list view items).N
This allows to access view items and menu items from Squish IDE's point & click interface. So verification points which check if, for example, a list view contains the expected items or a menu's item is checked as expected can be easily inserted.
This section discusses how to create and work with screenshot verification points.
Another common type of verifications is to compare the the visual output of a widget (or set of widgets) with an expected image. This means a screenshot of the desired widget(s) is taken and compared to a screenshot which has been taken when the test was created.
Generally it is advisable to use object property comparisons whenever possible to compare actual values and not screen output. The reason for this is that due to different resolutions and platforms, esp. changing fonts often lead to differing images while the actual content would be correct. This way tests will fail even though the output is correct.
A case where screenshot comparisons are a good solution is when comparing that a graph or diagram is drawn correctly.
Inserting such a screenshot verification point in Squish is just as easy as inserting an object property comparison. The steps to it are the same as described in Inserting a Verification Point (Section 5.2.1). Only instead of selecting object properties, in the right pane of Spy you have to click on the tab page Screenshots.
Now when marking objects in the left pane as checked, a screenshot will be taken and a preview of it will be placed in the right pane:
When now inserting the verification point, the taken screenshots will be saved as the expected results. When later executing this verification point in the test script, screenshots of the same widgets will be taken during the test run and compared against the saved screenshots.
In case a screenshot verification fails, Squish saves the new and differing image for later inspection. The Squish IDE also allows to view the differences between the expected and new image in the test log view via Show Image Diff.
The differences are shown in different ways such as a subtraction of the images, a side-by-side view and an animation where the expected and new result are displayed quickly one after the other in a loop making differences visible. It then is also possible to accept the new image as new expected result.
Squish provides a tool which allows to make screenshot verifications more robust. This is done using image masks which allow to specify areas in the screenshot which should be ignored or respected in the comparison.
Image masks can be created and modified for a screenshot in the verification point editor in the Squish IDE There it is possible to insert, modify and remove positive and negative masks.
While inserting verification points can be done without writing a single line of script code, it is also possible to code them in script. This is specially useful when a verification point needs to be more flexible, such as iterating over all items of a list view.
It never hurts to initially create all verification points using the point & click interface in the Squish IDE. Such a verification point is always saved as an external file, and a script line such as
test.vp("<VP-name>")
To convert such a statement into pure script code, you can right click on this statement in the Squish IDE and choose in the context menu. This way, for example, the line
test.vp("VP1")
becomes
# Verification Point 'VP1'
test.compare(findObject(":Addressbook.ABCentralWidget1.QListView1").childCount, 1)
test.compare(findObject(":Addressbook.ABCentralWidget1.QListView1.item_1").text0, "Max")
test.compare(findObject(":Addressbook.ABCentralWidget1.QListView1.item_1").text3, "max@mustermann.net")
test.compare(findObject(":Addressbook.ABCentralWidget1.QListView1.item_1").text2, "Bakerstreet 55")
test.compare(findObject(":Addressbook.ABCentralWidget1.QListView1.item_1").text1, "Mustermann")
To scriptify all verification points of a file, you can right click somewhere into the file and choose .
For more details about the script API for test statements, have a look at Test Statements (Section 16.1.6). Here is a small example in JavaScript which shows a dynamic verification point which iterates over all items of a Qt list view and checks that the item text is not empty. Such a verification point can't be created using the Squish IDE since the number of items is unknown and can be different on each test run. Such flexibility is only provided by using script languages:
function main() {
# get the list view
var listview = findObject("Addressbook.addressList");
# get the first item of the list view
var item = listView.firstChild();
var cnt = 0;
# loop as long as there is a valid item
while (!isNull(item)) {
cnt++;
# check that the text of column 0 is not empty
test.compare(item.text(0).isEmpty(), false);
# go to next item
item = item.nextSibling();
}
# store in the test results the number of checked items
test.log("Checked " + cnt + " items");
}