Test development workflow

To integrate tests into a platform continuous testing service, they should meet the guidelines on this page and follow this recommended flow.

  1. Use the Soong build system for Simple Test Configuration.
  2. Employ Test Mapping to create pre- and post-submit test rules directly in the Android source tree.
  3. Run tests locally using Atest.

Test types

Supported test types are:

Functional tests make assertions of pass or fail on test cases, while metrics tests generally perform an action repeatedly to collect timing metrics.

With standardized input/output format, the need for customized result parsing and post-processing per test is eliminated, and generic test harnesses can be used for all tests that fit into the convention. See the Trade Federation Overview for the continuous test framework included with Android.

Test case guidelines

Test cases executed through the continuous testing service are expected to be hermetic, meaning that all dependencies are declared and provided with the tests. See Hermetic Servers on the Google Testing Blog for an understanding of this principle. In short, hermetic tests require no:

  • Google Account sign-in
  • Connectivity configured (telephony/Wi-Fi/Bluetooth/NFC)
  • Test parameters passed in
  • Setup or tear down performed by test harness for a specific test case