Android platform testing

AOSP provides several tools and test suites for testing various parts of your implementation. Before continuing with this section, you should be familiar with the following terms:

Android-compatible device
A device that can run any third-party app written by third-party developers using the Android SDK and NDK. Android-compatible devices must adhere to the requirements of the Compatibility Definition Document (CDD) and pass the Compatibility Test Suite (CTS). Android-compatible devices are eligible to participate in the Android ecosystem, which includes potential licensure of the Google Play Store, potential licensure the Google Mobile Services (GMS) suite of app and APIs, and use of the Android trademark. Anyone is welcome to use the Android source code, but to be considered part of the Android ecosystem, a device must be Android compatible.
artifact
Artifacts are build-related logs that enable local troubleshooting.
Compatibility Definition Document (CDD)
A document that enumerates the software and hardware requirements for an Android-compatible device.
Compatibility Test Suite (CTS)

A free, commercial-grade test suite, available for download as a binary or as source in AOSP. The CTS is a set of unit tests designed to be integrated into your daily workflow. The intent of CTS is to reveal incompatibilities, and ensure that the software remains compatible throughout the development process.

CTS and platform tests aren't mutually exclusive. Here are some general guidelines:

  • If a test is asserting correctness of framework API functions or behaviors, and it should be enforced across OEM partners, it should be in CTS.
  • If a test is intended to catch regressions during platform development, and might require privileged permission to carry out, and might be dependent on implementation details (as released in AOSP), it should be a platform test.
Google Mobile Services (GMS)

A collection of Google apps and APIs that can be preinstalled on devices.

GoogleTest (GTest)

GTest is a C++ testing and mocking framework. GTest binaries typically access lower-level abstraction layers or perform raw IPC against various system services. The testing approach for GTest is usually tightly coupled with the service being tested. CTS contains the GTest framework.

instrumentation test

An instrumentation test provides a special test execution environment as launched by the am instrument command, where the targeted application process is restarted and initialized with basic application context, and an instrumentation thread is started inside the application process virtual machine. CTS contains instrumentation tests.

Logcat

Logcat is a command-line tool that creates a log of system messages, including stack traces of when the device throws an error and messages that you have written from your app with the Log class.

logging

Logging refers using a log to keep track of computer system events, such as errors. Logging in Android is complex due to the mix of standards used that are combined in the Logcat tool.

postsubmit test

Android postsubmit tests are performed when a new patch is committed to a common kernel branch. By entering aosp_kernel as a partial branch name, you can see a list of kernel branches with results available. For example, results for android-mainline can be found at https://ci.android.com/builds/branches/aosp_kernel-common-android-mainline/grid.

presubmit test

Presubmit tests are used to prevent failures from being introduced into the common kernels.

Trade Federation

Trade Federation, also called Tradefed, is a continuous test framework designed for running tests on Android devices. For example, Tradefed is used to run Compatibility Test Suite and Vendor Test Suite tests.

Vendor Test Suite (VTS)

The Android Vendor Test Suite (VTS) provides extensive capabilities for Android testing, promotes a test-driven development process, and automates HAL and OS kernel testing.

Platform test types

A platform test typically interacts with one or more of the Android system services or hardware abstraction layer (HAL) layers, exercises the functionalities of the subject under test, and asserts correctness of the testing outcome. A platform test might:

  • (type 1) Exercise framework APIs using Android framework. Specific APIs being exercised can include:
    • Public APIs intended for third-party apps
    • Hidden APIs intended for privileged apps, namely system APIs or private APIs (@hide, orprotected,package private`)
  • (type 2) Invoke Android system services using raw binder or IPC proxies directly.
  • (type 3) Interact directly with HALs using low-level APIs or IPC interfaces.

Type 1 and 2 tests are typically instrumentation tests, while type 3 tests are usually GTests.

What's Next?

Following is a list of the next documents that you might read: