“In less than 3 man-months I was able to come up-to-speed on both Squish and Python and replace our in-house solution, which took 2 man-years to build. We now have a higher quality product because of our automated testing with Squish.” said Marc Seter, CTO at Selden Integrated Systems, after having introduced Squish.
We had the pleasure to interview Marc Seter who is responsible for high- and low-level system design, project management, and selecting third-party tools. Since he also coordinates closely with their Quality Assurance Manager, he was the driving force behind introducing Squish a couple of months ago to replace their existing manual and home-brewed automated GUI tests with Squish.
Marc decided to go for Squish after having read an article on Squish in Trolltech’s Qt Quarterly magazine and got a recommendation for Squish from Trolltech. “Trolltech provided us with Qt, which is a powerful 3rd party tool that provided great value as a building block for our products. Effectively, they recommended Squish to us and because they’ve come through for us several times in the past we trusted their recommendation.”, he says.
“Squish’s documentation & examples are very good. In less than 3 man-months I was able to come up-to-speed on both Squish and Python and replace our in-house solution, which took 2 man-years to build.”, Marc adds after having automated a fair amount of their manual test cases using Squish.
Scripting and Customization
To create tests, you don’t need to manually write script code though. “You can quickly & easily record test scripts using Squish. The generated scripting code is simple, straight-forward and easily extensible. Running those test scripts is blazingly fast.” Marc adds “Also Squish’s IDE allows easy insertion of verification points.”
Marc’s team has been able to easily add their own customizations using Python, including implicit synchronization points which are critical when testing client/server applications. To underline the importance of using a full-blown scripting language for testing he adds “In every testing effort in which I’ve been involved, there’s been some squirrelly control path that pushed past the limitations of the automated testing tool. Being able to use Python’s extensive class library from within Squish test scripts to check POP3 email accounts, exercise HTML pages & forms, etc. have proved invaluable in building a comprehensive Quality Assurance solution.”
Besides the extensive scripting support, one of Squish’s big benefits is the tight integration of Qt and the cross-platform ability which allows to run the same test scripts on multiple platforms. “We’ve isolated & eliminated numerous bugs that were very elusive under one platform but consistently reproducible (and therefore easier to track down) under another.” Marc says “Our primary platform is Microsoft Windows (98/ME/NT/2000/XP). We also maintain a Linux version of iKE and intend to release iKE for Mac OS X. We run Squish-based automated testing on both our Linux and MS-Windows versions of iKE.”
iKE has a client/server architecture. So they took advantage of Squish’s ability to run and control multiple applications from one test script: “As a client/server application, we have several cases in which we need to run multiple clients and confirm that they’re synchronizing their data.”
About froglogic’s support Marc says “The froglogic support team is extremely knowledgeable about their product, about Qt, and how their customers use both. They’ve also demonstrated a broad depth of understanding in software development, operating systems, etc. Their responses to my email have been very timely, courteous, and helpful. Their primary objective is always to make sure that whatever’s preventing me from getting my work done is eliminated – regardless of where the problem’s located.”
The Results and The Future
Before using Squish, Selden used a combination of manual testing and their in-house implementation of an automated test solution. For their 2005.02.13 iKE release, they had a 3-tiered manual testing team consisting of 3 “tier-1” testers, 2 “tier-2” testers and “tier-3” testing was covered by their development team. Their testing cycle took about 6 weeks to complete.
After the introduction of Squish, the situation changed: “We were able to complete this testing cycle with half the resources, in 3/4 the time. Our Squish-based automated test solution is easier to setup, run and maintain than either our previous manual OR automated testing solutions. All in all, we have a higher quality product for this release of iKE [the first release which was tested using Squish] because of our automated testing.”
Marc also says that they are using their own iKE tool to manage their QA testing cycles, bug tracking, project management, developer time tracking, customer relationship management, and more. “Squish and iKE work very nicely together to provide us with an efficient QA environment.”
Looking ahead he says “Currently, we are continuing to automate test cases that we used to complete with a team of manual testers; our current Squish-based automated test covers approximately twice the number of test cases our previous automated solution covered. I estimate that, within the next three months, we will have replaced 97% of our current manual test cases with automated Squish-based solutions. Once we have automated as much of our manual test cases as feasible, we intend to setup an automated regression testing environment.”
When we asked Marc whether he would recommend Squish to others he answered “Absolutely! Keep up the great work!”
Selden Integrated Systems is a software solutions provider. They specialize in “remote office” technologies – work-flow automation, online collaboration, secure file sharing & instant messaging, and remote project management. Using their robust client/server architecture, they provide customized solutions to companies that need to increase their efficiency. They also provide tools that help customers streamline their own work-flow processes.