Webinar Q&A: How to Get Faster, More Reliable Automated Testing Results Using Continuous Integration

Webinar Q&A: How to Get Faster, More Reliable Automated Testing Results Using Continuous Integration

With our friends from QASource, we recently hosted a webinar, “How to Get Faster, More Reliable Automated Testing Results Using Continuous Integration,” where we showed you how to integrate your Squish tests with the Jenkins Continuous Integration server for automatic execution of your GUI tests in a CI pipeline.

We received lots of great questions during the live webinar Q&A, but did not get to answer all of them. The following list comprises the most frequently asked questions and their answers.

Q: Which type of job is preferred for real-time implementation – freestyle or pipeline jobs?

We recommend pipeline jobs over freestyle jobs, as they offer greater flexibility and stability in complex workflows. More information on the advantages of pipeline workflows can be found here.

Q: Is the initiation of tests after a code check-in possible only with using Git, or can other versioning tools be used?

Yes, other Version Control Systems can be used, for example, Mercurial.

Q: Can we distribute test executions across various nodes through Jenkins?

Absolutely. One can make use of the ‘parallel’ function in the pipeline script to distribute tests across available nodes. The following pseudo-code runs two suites, “foo” and “bar” in parallel:

parallel {
     'suite_foo': node {
         squish([testsuite: 'suite_foo'])
     }
     'suite_bar': node {
         squish([testsuite: 'suite_bar'])
     }
}

Q: Is Travis also supported? Do you provide a plugin for Squish integration?

Currently froglogic does not offer a plugin for integration with the Travis CI system, however running tests in batch can be achieved through command line calls to squishrunner. We plan to offer a plugin to Travis in the future. 

Q: Can we upload a maintainable file to the Squish for Jenkins plugin which specifies the tests to be run?

Not directly – however this can be accomplished with Jenkins pipeline scripts. Jenkins provides functions for reading files, splitting strings and iterating lists. The Squish plugin, when used in a pipeline, is just a function call with arguments. That means it is possible to use a Jenkins function for reading a file, converting the contents into a list and passing that as the corresponding argument for the Squish plugin to run only the given list of test cases. 

In pseudo-code, this might look like:

def testcases = readFile("testCaseList.txt").split("\n").join(" ")
def squish result = squish( [
           testsuite: "suite_foo",
           testcase: testcases])

The same idea could be used to read the list of test suites.

Q: Due to company compliance and security reasons, we cannot use open source tools for our CI workflows. Which commercial tools do you support for setting up a CI pipeline with Squish?

Squish supports several CI tools, not just Jenkins. Some of these include TeamCity, Ant, Maven and Bamboo. For more information on Squish’s extensive integration options, visit the corresponding features page.

Q: What is the long-term viability of tests run in Jenkins? For example, if after a year or so there are new versions of Squish and Jenkins, do I need to update the tests?

We strive towards keeping existing tests running with new versions of Squish. Older tests, even more than a year old, should continue to work as expected. 

Q: For the pipeline script, can we configure it such that tests are skipped if there is a failure?

Within the Squish plugin, you can configure your job to report a build as unstable if there are test failures. If there is a failure, the remaining tests will execute. Optionally, you can abort a build if a test failure is detected. 

Q: We have setup Squish with Jenkins, and now we are planning to execute tests on different machines and browsers for comprehensive coverage. What approach would you use to set up such an environment?

You can configure Jenkins to execute the same test suite on multiple machines in parallel. Your setup can include using different browsers to achieve comprehensive coverage. This is possible all within a properly written pipeline script.

Q: The object attributes of my application change every time I run the automation suites of my application. How can Squish handle dynamic object attributes for every test execution?

Squish can handle dynamic object attributes using easy-to-define Wildcard or Regular Expression matching. With these tools, one can edit the Object Map entry for dynamic objects to allow for inexact matches in object names. More information can be found in our documentation.

0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*