8.4. Making the Test More Robust

8.4.1. Factors Influencing Robustness
8.4.2. Coping With Timing Differences
8.4.3. Handling Different Page States

8.4.1. Factors Influencing Robustness

It's possible that you noticed already during this tutorial - the test script is quite fragile, especially due to various timing issues. Web applications are very asynchronous in that there are often phases in which new pages, images or other data is loaded and anything which executes actions on the web page needs to wait on that new data to arrive before it can continue. This anything is usually a user visiting the web site with a web browser, but it can just as well be a test script steered by Squish.

Also, certain technologies like cookies allow a web page to store some certain state. For instance, a web page could use cookies to memorize what you typed into some input field on your last visit, or which check boxes you checked. Hence, a client of a web page - and in our case, the test script is the client - must be prepared that a web page is in a different state when the script is executed than it was when the script was recorded.

Hence, for this tutorial, we will make the script more robust in two respects:

  1. We will make the script more robust to different timing issues by not using snooze all the time. Instead we will wait for certain objects to become accessible before proceeding with the execution of the script so that we know that those objects exists.

  2. For some cases, we will check the state of the objects on the web page before changing it. In particular, we will check whether any of the check boxes already has the desired state (checked or unchecked) before toggling it.

8.4.2. Coping With Timing Differences

Let's start with the timing issues. First, we will have a look at our test script again and check which snooze statements are causing the fragility:

function main()
{
    snooze(1.5);
    loadUrl(":http://validator.w3.org");
    snooze(5.6); 
    clickLink(":{tagName='A' innerText='Extended Interface'}");
    snooze(2.2); 
    setContext(":detailed.html");
    setFocus(":{tagName='INPUT' id='uri' name='uri' type='text'}");
    snooze(8.4);
    setText(":{tagName='INPUT' id='uri' name='uri' type='text'}", "http://www.froglogic.com");
    snooze(2.0);
    selectOption(":{tagName='SELECT' id='doctype' name='doctype' type='select-one'}", "HTML 4.01 Transitional");
    snooze(2.1);
    clickButton(":outline_checkbox");
    snooze(0.7);
    clickButton(":{tagName='INPUT' id='verbose' name='verbose' type='checkbox' value='1'}");
    snooze(0.6);
    clickButton(":{tagName='INPUT' type='submit' value='Validate this page'}");
    snooze(10.5); 
    setContext(":check");
    snooze(0.4);
    test.vp("VP1");
    closeWindow(":[Window]");
}

The critical snooze statements were marked with little numbers:

This snooze statement could possibly fail since loading the original URL (http://validator.w3.org) as done by the loadUrl call above could take longer than the time we're waiting (longer than 5.6 seconds in this case). If it takes longer, then the clickLink invocation below would fails since there is no Extended Interface link to click on yet.

Here we have another dangerous snooze; the clickLink call above takes us to a new page (the one with the extended interface, http://validator.w3.org/detailed.html but loading that new page could take longer than 2.2 seconds. If it would indeed take longer, then any of the next function calls (for instance the setFocus, setText or clickButton calls) would fail since the objects which they're operating on don't exist yet.

Finally, a third fragile snooze: the clickButton invocation above clicks the Validate this page button, which makes the server generate a report and redirect us to that new report page. This report generation, and downloading the HTML page to the web browser, could take longer than 10.5 seconds. If it would take longer, then our new verification point statement would fail since it tries to check the properties on a page which was not loaded yet.

As you can see, snooze statements are usually critical when used for waiting for a new page to load. Luckily, there's a simple cure for all of these issues: the waitForObject function.

Just like the snooze function, the waitForObject call suspends the execution of the test script. However, it does not wait for a specific duration but it waits for a given object to exist before allowing the script to continue.

[Tip]Tip

There are even more methods for introducing such synchronization points into the script, to make sure that the application being tested and the script are always synchronized. You can learn more about synchronization of scripts in the section How to Create and Use Synchronization Points (Section 15.1.9) and a special function to wait for a web page to be loaded completely before accesing it at How to Synchronize Web Page Loading for Testing (Section 15.1.5.7).

We could resolve the fragility of our test script with respect to the timing by replacing the three snooze statements marked above with adequate waitForObject calls which wait until some object contained in the new page exists. For instance, let's take the first fragile fragment of our script:

    [..]
    loadUrl(":http://validator.w3.org");
    snooze(5.6);
    clickLink(":{tagName='A' innerText='Extended Interface'}");
    [..]

We load the validator homepage, hope that it's done after 5.6 seconds, then click on the Extended Interface link. We could make this more robust by replacing the snooze with a waitForObject call which waits for the link we want to click on with clickLink to exist, as follows:

    [..]
    loadUrl(":http://validator.w3.org");
    waitForObject(":{tagName='A' innerText='Extended Interface'}");
    clickLink(":{tagName='A' innerText='Extended Interface'}");
    [..]
[Note]Note

This waitForObject call would not wait forever. In fact, it would wait twenty seconds at most. If the object does not exist after twenty seconds, then the waitForObject returns an error (see How to Create and Use Synchronization Points (Section 15.1.9)).

This is much better already. Similarly, we replace the other two snooze calls with waitForObject calls which wait for some object to exist. You can see the resulting improved script here:

function main()
{
    snooze(1.5);
    loadUrl(":http://validator.w3.org");
    waitForObject(":{tagName='A' innerText='Extended Interface'}"); 
    clickLink(":{tagName='A' innerText='Extended Interface'}");
    waitForObject(":{tagName='INPUT' id='uri' name='uri' type='text'}"); 
    setContext(":detailed.html");
    setFocus(":{tagName='INPUT' id='uri' name='uri' type='text'}");
    snooze(8.4);
    setText(":{tagName='INPUT' id='uri' name='uri' type='text'}", "http://www.froglogic.com");
    snooze(2.0);
    selectOption(":{tagName='SELECT' id='doctype' name='doctype' type='select-one'}", "HTML 4.01 Transitional");
    snooze(2.1);
    clickButton(":outline_checkbox");
    snooze(0.7);
    clickButton(":{tagName='INPUT' id='verbose' name='verbose' type='checkbox' value='1'}");
    snooze(0.6);
    clickButton(":{tagName='INPUT' type='submit' value='Validate this page'}");
    waitForObject(":DOCUMENT.HTML1.BODY1.DIV2.DIV1.FORM1.TABLE1.TBODY1.TR1.TD1"); 
    setContext(":check");
    snooze(0.4);
    test.vp("VP1");
    closeWindow(":[Window]");
}

As you can see, we replaced all three fragile snooze statements with equivalent waitForObject calls (in the lines marked with the numbers).

[Note]Note

In case you wonder where the long object name comes from which we used for the last waitForObject call: we want to wait for an object which is used by the verification point check executed by the test.vp function. To find out which object(s) are needed by the verification point, we scriptify it which means we're replacing the test.vp code with the actual script code executed behind the scenes. Doing that is explained in the section How to Create and Use Verification Points in Test Scripts (Section 15.3.3)

8.4.3. Handling Different Page States

Our script is much more robust in the face of different network response times already, but we still don't cope with possibly changed page states. As mentioned earlier, technologies which memorize the page state (such as the so called cookies) are often used by web applications to make the interface more usable to the user. For instance, online web shops often memorize your user name and password when you enter them for the first time, and then fill them into the corresponding input fields when you visit the website for a second time. Similarly, it's possible that the validator page we're testing remembers which elements of the web application we chose, which means that the application is not in the same state when we run the test as it has been when the test was created.

Luckily, only two calls in our test script can be influenced by the page state: the two clickButton invocations which click on the No attributes and Verbose output check boxes. The problem is that the clickButton function merely clicks a button (this includes check boxes, radio buttons and more - see Web Object API (Section 16.1.9)), and just that. In the case of a checkbox though, clicking it means toggling it. Unfortunately, that's not what we intended to do. We wanted to enable the No attributes checkbox and disable the Verbose output checkbox. Hence, we shouldn't always toggle the button but only if it's currently not in the checked/unchecked state we intend.

This is fairly easy to implement, too. Squish lets you access virtually any property of every HTML element of your web application (see How to Access Web Object Properties (Section 15.1.5.3)). As an example. let's rewrite the first error-prone clickButton invocation. Here's the current code:

    [..]
    clickButton(":outline_checkbox");
    [..]

We start rewriting that by adjusting the code so that clickButton doesn't take the name of the object we want to click directly. Instead, we use the findObject function to acquire the object, and then click that:

    [..]
    var outlineCheckbox = findObject(":outline_checkbox");
    clickButton(outlineCheckbox);
    [..]

So far so good. Just like all other functions which compose the convenience API for web applications (see Web Object API (Section 16.1.9)), the clickButton can take the name of the object to operate on as a string as well as the object itself as found by the findObject function.

Now all that's missing is a check for the checked property. The clickButton call should only be executed if it's not checked yet. Here's how to do it:

    [..]
    var outlineCheckbox = findObject(":outline_checkbox");
    if (!outlineCheckbox.checked)
        clickButton(outlineCheckbox);
    [..]

Not too difficult, was it? That's all we need for making a clickButton invocation on a checkbox robust in the face of changing page state. If we do the same for the other checkbox, we end up with our revised script which is now much more robust when it comes to changing page state or different network response times:

function main()
{
    snooze(1.5);
    loadUrl(":http://validator.w3.org");
    waitForObject(":{tagName='A' innerText='Extended Interface'}");
    clickLink(":{tagName='A' innerText='Extended Interface'}");
    waitForObject(":{tagName='INPUT' id='uri' name='uri' type='text'}");
    setContext(":detailed.html");
    setFocus(":{tagName='INPUT' id='uri' name='uri' type='text'}");
    snooze(8.4);
    setText(":{tagName='INPUT' id='uri' name='uri' type='text'}", "http://www.froglogic.com");
    snooze(2.0);
    selectOption(":{tagName='SELECT' id='doctype' name='doctype' type='select-one'}", "HTML 4.01 Transitional");
    snooze(2.1);
    var outlineCheckbox = findObject(":outline_checkbox");
    if (!outlineCheckbox.checked) {
        clickButton(outlineCheckbox);
        snooze(0.7);
    }
    var verboseCheckbox = findObject(":{tagName='INPUT' id='verbose' name='verbose' type='checkbox' value='1'}");
    if (verboseCheckbox.checked) {
        clickButton(verboseCheckbox);
        snooze(0.6);
    }
    clickButton(":{tagName='INPUT' type='submit' value='Validate this page'}");
    waitForObject(":DOCUMENT.HTML1.BODY1.DIV2.DIV1.FORM1.TABLE1.TBODY1.TR1.TD1");
    setContext(":check");
    snooze(0.4);
    test.vp("VP1");
    closeWindow(":[Window]");
}