GUI Testing Through Optical Character Recognition in the Cloud

GUI Testing Through Optical Character Recognition in the Cloud

Robust and reliable GUI tests are essential for the continuous deployment of today’s software projects. The Squish GUI Tester supports you in achieving this goal through dedicated Squish editions, which are aware of various GUI toolkits and technologies. Image-based Testing is included as a complementary feature for dynamically rendered visualizations. Since Squish 6.5, these test automation approaches are joined by AI-driven Optical Character Recognition (OCR). This article focuses on the configuration of Squish needed to benefit from AI-driven Optical Character Recognition (OCR) Cloud Services.

Optical Character Recognition (OCR)

You might find yourself in a situation where text should be recognized but dedicated GUI object recognition does not do the job. Custom rendering or usage of unsupported third-party toolkits could be the reason. You could fall back to the image recognition approach in these cases. This requires you to create a search image and the identification would depend on the text rendering. This can be error-prone for rendered text due to anti-aliasing, scaling or font substitution.

Luckily, the Squish GUI Tester introduced Optical Character Recognition as part of its 6.5 release. This feature enables you to recognize text on screen, no matter how it is rendered.

Tesseract is the OCR engine that is currently supported by Squish for local installation. But you might want to leverage a cloud service for your enterprise needs. Amazon Rekognition is probably the most popular cloud OCR service. It is really easy to integrate with Squish as we’ll show below. Squish also supports the OCR.Space API as an alternative cloud service, as you will see afterwards.

Amazon Rekognition OCR Cloud Service

The following steps have to be done to enable Squish Optical Character Recognition through Amazon Rekognition:

  1. AWS account Setup

Setup an AWS account, which includes free image recognition for 6000 images per month within the first 12 months.

  1. Create an IAM User and AWS access key

Authentication for the AWS cloud service access is done through an access key. This key will be generated as part of the IAM user setup with AWS administrator permissions. Make sure to download the CSV file with the access key when creating your IAM user. Forgot to do this? Simply create a new key through the IAM Console.

The Squish GUI Tester already includes the glue to connect to the cloud service. There is no need to install any SDK as suggested by the AWS documentation.

  1. Configure Squish to use Amazon Rekognition

Finally, open the Squish Preferences (Edit|Preferences) and navigate to the Squish OCR engine settings. Switch the OCR Engine to Amazon Rekognition and enter the key ID (API key ID) and access key (API key) as well as the preferred Region of the Amazon datacenter. Once you have applied these changes, Squish will use Amazon‘s cloud service to identify any text within your test without any further configuration. You can easily switch to a different OCR engine at a later point without changing the test itself.

Squish OCR engine settings

OCR.Space Cloud Service

OCR.Space is another Optical Character Recognition cloud service offering. It includes 25 000 calls per month for free and there are commercial plans which include even more calls. If you prefer a self-hosted solution, don’t miss their OCR.Space Local on-premise server option.

The configuration is straightforward:

  1. Request an OCR.Space API key

Request an OCR.Space API key in a few steps by creating an account.

  1. Configure Squish to use the OCR.Space cloud service

As for Amazon Rekognition, simply open the Squish Preferences (Edit|Preferences) and navigate to the Squish OCR settings to enter the API key you received by email. Make sure to switch the OCR Engine setting to OCR.Space.

Looking for a different OCR Engine?

The OCR support in Squish has been designed to support various OCR engines. Tesseract, Amazon Rekognition and OCR.Space are the first supported engines. If you are missing any specific engine, don’t hesitate to let us know. We would be pleased to look into integrating your preferred engine with Squish.

Automate Tests Based on OCR and Image Recognition

Our documentation covers the topic of test automation with OCR and Image Recognition in detail. And don’t miss the recordings of our release webinars on OCR and Image Recognition for a quick feature demo:

Florian joined froglogic in 2011 after working as a Software Developer and QA Engineer. Being a froglogic Consultant, he conducted dozens of Squish trainings, provided consulting services to froglogic clients, and gained experience working with QA departments across all industries. These days, his focus is on Technical Sales.

0 Comments

Leave a reply

电子邮件地址不会被公开。 必填项已用*标注

*