To integrate tests into a platform continuous testing service, they should meet the guidelines on this page and follow this recommended flow.
- Use the Soong build system for Simple Test Configuration.
- Employ Test Mapping to easily create pre- and post-submit test rules directly in the Android source tree.
- Run tests locally using Atest.
Supported test types are:
- Instrumentation tests support both functional and metrics tests. See Test your app for general app testing guidance.
- Native tests support these types:
- JAR host tests using JUnit
Functional tests make assertions of pass or fail on test cases, while metrics tests generally perform an action repeatedly to collect timing metrics.
With standardized input/output format, the need for customized result parsing and post-processing per test is eliminated, and generic test harnesses can be used for all tests that fit into the convention. See the Trade Federation Overview for the continuous test framework included with Android.
Test case guidelines
Test cases executed via continuous testing service are expected to be hermetic meaning all dependencies are declared and provided with the tests. See Hermetic Servers on the Google Testing Blog for an understanding of this principle. In short, hermetic tests require no:
- Google account sign-in
- connectivity configured (telephony/Wi-Fi/Bluetooth/NFC)
- test parameters passed in
- setup or tear down performed by test harness for a specific test case