Squish tip of the week: How to produce more accurate test results

Squish tip of the week: How to produce more accurate test results

To produce accurate and meaningful test results, each test should be able to run alone and as part of a test suite.

What if my test requires my application be in a specific state to run?

The key of your test is to determine if a particular requirement is met or result is produced. The path to setting up the scenario in the application where the test can be performed should not dictate if the test itself passed or failed.

How can I separate the two?

Incorporating setup and teardown steps outside the main test ensures test results indicate the status of the actual requirement or feature targeted.

Use Squish’s init() and cleanup() functions in addition to the main() function to break the test apart:

def init():
    try:
        # setup steps to execute prior to main test
        test.log("trying some aut setup")
    except Exception, e:
        test.fatal("Test setup failed. Main will not" +
                  " run.", "See 'Script Error' details.")
        raise Exception(e)

def main():
    # main executes only if init() finished without issue
    startApplication("AddressBookSwing.jar")
    activateItem(waitForObjectItem(":Address Book_javax"
                            + ".swing.JMenuBar", "File"))
    test.log("does it get here?")

def cleanup():
    # cleanup steps to execute after to main test,
    # even if main fails or never executes
    test.log("cleaned up from test case")

Sample Report

Summary

Test Cases 1
Tests 0
Passes 0
Fails 0
Errors 1
Fatals 1

Results

tst_general Thu Oct  2 13:34:49 2014
FATAL Test setup failed; main() will not execute. See ‘Script Error’ for details.
ERROR Script Error Exception: Item ‘Filessss’ in object ‘:Address Book_javax.swing.JMenuBar’ not found or not ready.
LOG Post test case cleanup complete

Ad-hoc testing or test randomization has also proven beneficial

Read more about test randomization here: Squish tip of the week: Alter test scenario workflow to increase test effectiveness

Read more in other Squish resources below: