Rebooting a Remote Test Environment From a Test Script

Rebooting a Remote Test Environment From a Test Script

Sometimes we can’t launch our application from our test case, which is often the case if we perform tests on embedded devices, where the AUT is launched during device startup.

In such cases, we simply attach to the AUT at the beginning of each test. At the same time, we want to keep our test cases independent, so bringing back the AUT to the initial state is part of the cleanup of the previous test case.

Unfortunately, if something unexpected happens during execution, our AUT might get stuck in an unknown state or become frozen. The only way to bring it back to the desired point may be to restart the device.

In the following Python example, we’re going to show you the example implementation of this approach.

The rebooting will happen in the init() function. This way we are independent of the previously executed test outcome.
We will use the paramiko Python module to establish an SSH connection and execute a reboot command on the device. The paramiko module is not part of the Squish Python package, but can be easily installed with pip.

  1. Define the init() function that is called automatically before a test case main() execution:
def init():
	test.startSection("Test initialization")
	attachToApplication("AUT")
	if is_initial():
		test.log("AUT is in the initial state. Continue test execution.")
	else:
		test.log("AUT is not in the initial state. Reboot test environment")
		reboot()	
		waitForAut(timeout_s=20)			
	test.endSection()

2. Define is_initial() function that checks if the AUT is in the initial state:

def is_initial(timeout_s=20):
    """ custom code that checks if your application is in the initial state """
    try:
        waitForObject(names.initial_screen, timeout_s*1000)
        return True
    except:
        return False

3. Define thereboot() function that connects to the embedded device over SSH and sends a reboot command:

import paramiko

def reboot():
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh.connect(hostname=DEVICE_IP, username=DEVICE_USERNAME, password=DEVICE_PASSWORD)
    ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("sudo shutdown -r now")
    if ssh_stdout.channel.recv_exit_status() > 0:
        raise RuntimeError("AUT reboot failed")

4. Define the waitForAUT() function that waits for the device to be rebooted and that the AUT is available for attaching again:

from datetime import datetime

def waitForAut(timeout_s=20):
    start = datetime.now()        
    while True:
        try:
            attachToApplication("AUT")
        except RuntimeError as e:
            if (datetime.now() - start).total_seconds() > timeout_s:
                raise RuntimeError("Can't attach to the AUT")
            else:
                pass

Please note that you can use a global cleanup() and init() to implement this solution.

Jakub has worked in Quality Assurance since 2012. Over the years he has gained experience in manual, automatic and performance testing. A Python enthusiast, Jakub joined froglogic in 2016 to support Squish users, conducting trainings and offering consulting.

2 Comments

  1. Mohsen Naghian 4 months ago

    Hi Jakub,
    In this example, the idea is to bring the AUT to a known stage so the next test case would not fail, and the proposed solution would work perfectly. How about if the test is actually testing weather under some specific condition an AUT is rebooting (for example testing the recovery from application crash)? In those scenario the application is intentionally crashed and the test is to check if it is coming up after some timeout. Of course one may increase the squish server timeout to not return exception if the AUT is not accessible, but if that timeout is not enough again the test would fail. Currently we use applicationContext.detach() before the expected AUT crash, and then similarly as you used waitForAUT, we wait for the application to come to life and so squish get attached to it.
    Is this a reasonable workaround, or is there some smarter way to do so?

    • Author
      Jakub Topolski 4 months ago

      Hi Mohsen,
      At first glance, your solution seems reasonable. In the proposed solution, Squish re-attaches to the AUT as soon as possible. In many cases, we can provide a much longer timeout than expected reboot time and still get results fast in case of successful execution.
      If you would like to discuss your doubts or more details I encourage you to contact us via squish@froglogic.com
      Jakub

Leave a reply

Your email address will not be published. Required fields are marked *

*