Sometimes we can’t launch our application from our test case, which is often the case if we perform tests on embedded devices, where the AUT is launched during device startup.
In such cases, we simply attach to the AUT at the beginning of each test. At the same time, we want to keep our test cases independent, so bringing back the AUT to the initial state is part of the cleanup of the previous test case.
Unfortunately, if something unexpected happens during execution, our AUT might get stuck in an unknown state or become frozen. The only way to bring it back to the desired point may be to restart the device.
In the following Python example, we’re going to show you the example implementation of this approach.
The rebooting will happen in the init() function. This way we are independent of the previously executed test outcome.
We will use the
- Define the
init()
function that is called automatically before a test casemain()
execution:
def init():
test.startSection("Test initialization")
attachToApplication("AUT")
if is_initial():
test.log("AUT is in the initial state. Continue test execution.")
else:
test.log("AUT is not in the initial state. Reboot test environment")
reboot()
waitForAut(timeout_s=20)
test.endSection()
2. Define is_initial()
def is_initial(timeout_s=20):
""" custom code that checks if your application is in the initial state """
try:
waitForObject(names.initial_screen, timeout_s*1000)
return True
except:
return False
3. Define thereboot()
import paramiko
def reboot():
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname=DEVICE_IP, username=DEVICE_USERNAME, password=DEVICE_PASSWORD)
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("sudo shutdown -r now")
if ssh_stdout.channel.recv_exit_status() > 0:
raise RuntimeError("AUT reboot failed")
4. Define the waitForAUT()
function that waits for the device to be rebooted and that the AUT is available for attaching again:
from datetime import datetime
def waitForAut(timeout_s=20):
start = datetime.now()
while True:
try:
attachToApplication("AUT")
except RuntimeError as e:
if (datetime.now() - start).total_seconds() > timeout_s:
raise RuntimeError("Can't attach to the AUT")
else:
pass
Please note that you can use a global cleanup() and init() to implement this solution.
2 Comments
Hi Mohsen,
At first glance, your solution seems reasonable. In the proposed solution, Squish re-attaches to the AUT as soon as possible. In many cases, we can provide a much longer timeout than expected reboot time and still get results fast in case of successful execution.
If you would like to discuss your doubts or more details I encourage you to contact us via squish@froglogic.com
Jakub
Hi Jakub,
In this example, the idea is to bring the AUT to a known stage so the next test case would not fail, and the proposed solution would work perfectly. How about if the test is actually testing weather under some specific condition an AUT is rebooting (for example testing the recovery from application crash)? In those scenario the application is intentionally crashed and the test is to check if it is coming up after some timeout. Of course one may increase the squish server timeout to not return exception if the AUT is not accessible, but if that timeout is not enough again the test would fail. Currently we use applicationContext.detach() before the expected AUT crash, and then similarly as you used waitForAUT, we wait for the application to come to life and so squish get attached to it.
Is this a reasonable workaround, or is there some smarter way to do so?