Retry isolation

See the Automatic Test Retry documentation to understand how the retry feature works.

When executing retries, if the device has entered into a bad state that does not allow tests to succeed anymore. Use the isolation feature restore the device into a fresh initial state and allow tests to run and succeed.

Enable the feature

The retry isolation can be enabled with the follow options in addition to the retry options:

--retry-isolation-grade <FULLY_ISOLATED|REBOOT_ISOLATED>

or disabled with:

--retry-isolation-grade NOT_ISOLATED

The feature is disabled by default.

Different grade of isolation

The isolation grade represents the extent of the isolation we want to configure between retries:

  • FULLY_ISOLATED will completely wipe & resetup the device by rerunning the configured target_preparers of the configuration
  • REBOOT_ISOLATED will reboot the device

Note on Compatibility Test Suite (CTS)

Android partner runs of CTS currently do not support FULLY_ISOLATED

What do the results look like?

Result reporters receive a marker that a given module or run was running in isolation and can elect to display the marker in their report.

  • At the module level the module-isolated property will be set to its isolation grade.
  • At the run level the run-isolated property will be set to its isolation grade.

For example: the first module running after being isolated will be reported as module-isolated:FULLY_ISOLATED.

A test run in isolation provides a strong signal. Be it pass or fail, the test owner should have high confidence that no bad state from a previous test was left on the device.