How to Locate an Object on a Map

How to Locate an Object on a Map

Motivation

Consider an application that features a search resulting in markers being shown on a map. We want to verify that an expected number of occurrences of a marker, a known image, is shown. Broadly speaking, we want to count how often a particular image is part of a bigger image.

test.imagePresent()

A way to tackle this is by utilizing a Squish API that lets us search for images in our AUT, the Google Maps website in this case. The verification function test.imagePresent() is one of them. The function simply logs a pass/fail depending on whether a given image is found. But this might not be flexible enough. In our case it is not.

waitForImage() and findImage()

For more flexibility, there are waitForImage() and findImage(). If in doubt, stick with the former: it gives the image in question some time to become visible.

The image we want to search through is referenced via an object name and needs to be extracted and stored in our test suite. To extract it, or, in other words, to take the marker from the rendered map, we use our favorite image editing tool to remove everything around the marker. The area around the marker should become transparent instead. This way, Squish ignores these extraneous parts when searching for the marker on the map.

Still, there are two things to watch out for. First, we should ensure that the alpha-transparent edge of the marker is removed as well if we aim for pixel-perfect matches. Second, we should be mindful that the markers contain the single characters A, B, etc. on top of them.

For the first consideration, we could accept the existence of small differences between the images by enabling tolerant search mode, a feature all image lookup APIs support. In case the default setting is not tolerant enough, we could override the default threshold for how much difference is accepted:

waitForImage("map_marker.png", {tolerant: 1, threshold: 0.95});

For the second consideration, where we want to exclude the alphabet characters, we again use our favorite image editing tool and remove the area that holds the character from our image and set it to transparent, to avoid it being used for detecting the marker on the map.

Left: Marker to be found. Right: Masked marker to improve search.

Solution

With these preparations, we can develop a Squish test that searches for the marker on the given image. By default it will return the first occurrence only. The search can be customized using the ‘occurrence’ parameter, though.

In a loop we ask findImage() for all occurrences of the marker; the property object passed to findImage() has the occurrence value increased by 1 on each iteration. Once no additional marker can be found, findImage() throws an exception. We use this as a signal to stop looking and have our checkMarkers() function return a list of found occurrences.

import { Wildcard } from 'objectmaphelper.js';

var googleBrowserTab = {"title": "Google", "type": "BrowserTab"};

var googleQText = {"container": googleBrowserTab, "form": "tsf",
        "name": "q", "tagName": "INPUT", "type": "text",
        "visible": true};

var karteVonPackstationIMG = {"img_alt":
        new Wildcard("Karte von packstation*"), "tagName": "IMG",
        "visible": true};

function main() {
    var expectedNoOfMarkers = 3;
    doGMapsSearch();
    checkMarkers(expectedNoOfMarkers);
}

function doGMapsSearch() {
    startBrowser("https://www.google.com");
    typeText(waitForObject(googleQText), "packstation hamburg");
    typeText(waitForObject(googleQText), "<Return>");
    waitForObject(karteVonPackstationIMG);
}

function checkMarkers(expectedNoOfMarkers) {
    var threshold = 95;
    var m = findMarkersOnMap(threshold);
    test.compare(expectedNoOfMarkers, m.length,
            "Expected exactly " + expectedNoOfMarkers + " markers");
}

function findMarkersOnMap(threshold)
{
    var foundMarkers = [];
    var occ = 0;
    try {
        while (true) {
            var marker = findImage("gmap_marker_masked.png",
                    {'tolerant': 1, 'threshold': threshold,
                        'occurrence': ++occ},
                    karteVonPackstationIMG);

            foundMarkers.push(marker);
        }
    } catch (e) {
        if (!(e instanceof TypeError)) {
            throw e;
        }
        // We expect this exception after all markers were found
    }
    return foundMarkers;
}

You might have noticed the usage of findImage() (in contrast to waitForImage()), even though earlier we suggested to stick with the latter. We waited for the map image to be loaded before entering the loop. Thus we can expect that it is not changing anymore. But waitForImage() would make multiple attempts to locate a (non-existing) fourth marker. Test execution would be thus slowed down unnecessarily.

Outlook

In this example, we cared about the number of markers only and masked the text that is visible on each marker. Verifying the text on each marker would be possible as well. For achieving that, Squish offers a built-in Optical Character Recognition (OCR) approach. The function getOcrText() could be used to extract the text displayed on each marker.

Roberto was born in Dresden, Germany. He studied at Dresden and Hamburg where he learned to program in Pascal and Delphi. In his spare time he taught himself web programming, and in particular HTML, CSS, JavaScript, and PHP. He studied mathematics and computer science in Hamburg, during which time he made contact with froglogic. Roberto joined froglogic in autumn 2006.

0 Avis

Laisser une réponse

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *

*