To integrate tests into a platform continuous testing service, they should meet the guidelines on this page and follow this recommended flow.
- Use the Soong build system for Simple Test Configuration.
- Employ Test Mapping to create pre- and post-submit test rules directly in the Android source tree.
- Run tests locally using Atest.
Test types
Supported test types are:
- Instrumentation tests support both functional and metrics tests. See Test your app for general app testing guidance.
- GoogleTest (GTest) supports the
following test types:
- Functional GTests using the GTest framework
- Metric tests using
google-benchmark
- JAR host tests using JUnit
Functional tests make assertions of pass or fail on test cases, while metrics tests generally perform an action repeatedly to collect timing metrics.
With standardized input/output format, the need for customized result parsing and post-processing per test is eliminated, and generic test harnesses can be used for all tests that fit into the convention. See the Trade Federation Overview for the continuous test framework included with Android.
Test case guidelines
Test cases executed through the continuous testing service are expected to be hermetic, meaning that all dependencies are declared and provided with the tests. See Hermetic Servers on the Google Testing Blog for an understanding of this principle. In short, hermetic tests require no:
- Google Account sign-in
- Connectivity configured (telephony/Wi-Fi/Bluetooth/NFC)
- Test parameters passed in
- Setup or tear down performed by test harness for a specific test case