Is Verifying Video Playback That Hard?

A common question posed in the software QA space is: how can I verify the playback of a video in my application? Some responses to this question include using 3rd party, expensive add-ons to existing automation software or require clunky workarounds. Here, we discuss the challenges in verifying video playback, and an easy way to work around them using the Squish GUI Tester.

Challenges in Video Verification

Verifying the complete video playback can be difficult. The complete video may be relatively large, and comparing a pre-captured video to one captured while replaying the test script can be very time consuming (the whole video first needs to be played in the test execution). More importantly, comparing such video sequences to one another can be extremely resource-intensive and brittle. Just a slight stutter in the playback due to external reasons (e.g., an anti-virus scan taking place during playback), can cause a time offset of the frames of the "golden master" video and the one captured at runtime, which would then require resynchronization of the frames to make the comparison tolerant to such effects.

A Simple, Scripted Workaround

An alternative to capturing the whole video for comparison is to let your test script take two or more screenshots while the video is playing. Then, the test script can compare these screenshots at runtime and report whether the screenshot images are identical or not.

The assumption that this is based on is that if the video is playing back as expected, it will be extremely unlikely that two screenshots of the video will ever have the same content. So while this approach does not verify that "my video" is being played, it verifies that "something is being played", which may be sufficient for many verification purposes.

All we need is the object name of the video player GUI control ("names.video_player"):

# Look up the video player control:
vidplayer = waitForObject(names.video_player)

# Grab screenshot #1 of the video player control:
scrshot1 = object.grabScreenshot(vidplayer)

# Wait a short moment to ensure that the video
# play back has progressed sufficiently:
snooze(1)

# Grab screenshot #2 of the video player control:
scrshot2 = object.grabScreenshot(vidplayer)

# Verify that the two screenshots are different:
test.verify(!scrshot1.equals(scrshot2))

Next Steps

  • The above solution could also be executed in a loop, to retry in case the two screenshots happen to be identical.
  • One may also want to attach the two screenshots to the test results as proof and for visualization, via test.attachImage().
  • Also, an alternative approach could be to let the application play a prefabricated video with well-known image sequences (for example showing all black in the first 10 seconds, and all white in the following 10 seconds), and then verify this via screenshot verification points.

Further Reading

For more on the functions listed above, take a look at our online docs:

For info on built-in verification point types available in Squish, visit here.

Comments

    The Qt Company acquired froglogic GmbH in order to bring the functionality of their market-leading automated testing suite of tools to our comprehensive quality assurance offering.