Initializing your Repo client
Follow Dowloading the Source
to get and build the Android source code, but specify a particular CTS branch
name, for example
-b android-5.0_r2, when issuing the
init command. This ensures that your CTS changes are included in the
next CTS release and beyond.
Building and running CTS
Execute the following commands to build CTS and start the interactive CTS console:
make cts -j32 TARGET_PRODUCT=aosp_arm64
In the CTS console, enter:
tf> run cts --plan CTS
Writing CTS tests
CTS tests use JUnit and the Android testing APIs. Review the
tutorial while perusing the existing tests under the
cts/tests directory. You will see that CTS tests mostly follow the same
conventions used in other Android tests.
CTS runs across many production devices, so the tests must follow these rules:
- Take into account varying screen sizes, orientations, and keyboard layouts.
- Use only public API methods. In other words, avoid all classes, methods, and fields that are annotated with the "hide" annotation.
- Avoid using view layouts or relying on the dimensions of assets that may not be on some device.
- Don't rely upon root privileges.
Test naming and location
Most CTS test cases target a specific class in the Android API. These tests
have Java package names with a
cts suffix and class names with a
Test suffix. Each test case consists of multiple tests, where each
test usually exercises a particular method of the class being tested.
These tests are arranged in a directory structure where tests are grouped into
different categories such as "widgets" and "views."
For example, the CTS test for the Java package
android.widget.cts.TextViewTest with its Java package name as
android.widget.cts and its class name as
- Java package name
The Java package name for the CTS tests is the package name of the class that the test is testing, followed by
.cts. For our example, the package name would be
- Class name
The class name for CTS tests is the name of the class that the test is testing with "Test" appended. For example, if a test is targeting
TextView, the class name should be
- Module name (CTS v2 only)
CTS v2 organizes tests by module. The module name is usually the second string of the Java package name (in our example,
The directory structure and sample code depend on whether you are using CTS v1 or CTS v2.
For Android 6.0 and lower, use CTS v1. For CTS v1, the sample code is at
The directory structure in CTS v1 tests looks like this:
cts/ tests/ tests/ package-name/ Android.mk AndroidManifest.xml src/ android/ package-name/ SampleDeviceActivity.java cts/ SampleDeviceTest.java
For Android 7.0 and higher, use CTS v2. For details, see the sample test in AOSP.
The CTS v2 directory structure looks like this:
cts/ tests/ module-name/ Android.mk AndroidManifest.xml src/ android/ package-name/ SampleDeviceActivity.java cts/ SampleDeviceTest.java
New sample packages
When adding new tests, there may not be an existing directory to place your test. In those cases, you need to create the directory and copy the appropriate sample files.
If you're using CTS v1, refer to the example under
cts/tests/tests/example and create a new directory. Also,
make sure to add your new package's module name from its
cts/CtsTestCaseList.mk. This makefile is used by
build/core/tasks/cts.mk to combine all the tests to create
the final CTS package.
Use the sample test
to quick start your new test module with following steps:
- Run this command to create the test directory and copy sample files to it:
mkdir cts/tests/module-name && cp -r cts/tests/sample/* cts/tests/module-name
- Navigate to
cts/tests/module-nameand substitute all instances of "[Ss]ample" following the recommended naming convention from above.
SampleDeviceActivityto exercise the feature you're testing.
SampleDeviceTestto ensure that the activity succeeds or logs its errors.
Other Android directories such as
res can also be added. To add JNI code,
create a directory in the root of the project next to
src with the native
code and an
Android.mk makefile in it.
The makefile typically contains the following settings:
LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libCtsSample_jni # don't include this package in any target LOCAL_MODULE_TAGS := optional LOCAL_SRC_FILES := list of source code files LOCAL_C_INCLUDES := $(JNI_H_INCLUDE) # Tag this module as a cts test artifact LOCAL_COMPATIBILITY_SUITE := cts LOCAL_SHARED_LIBRARIES := libnativehelper LOCAL_SDK_VERSION := current include $(BUILD_SHARED_LIBRARY)
Finally, modify the
Android.mk file in the root of the
project to build the native code and depend on it, as shown below:
# All tests should include android.test.runner. LOCAL_JAVA_LIBRARIES := android.test.runner # Includes the jni code as a shared library LOCAL_JNI_SHARED_LIBRARIES := libCtsSample_jni # Include for InstrumentationCtsTestRunner LOCAL_STATIC_JAVA_LIBRARIES := ctstestrunner... LOCAL_SDK_VERSION := currentinclude $(BUILD_CTS_PACKAGE) #Tells make to look in subdirectories for more make files to include include $(call all-makefiles-under,$(LOCAL_PATH))
Fixing or removing tests
In addition to adding new tests, you can fix or remove tests annotated with "BrokenTest" or "KnownFailure."
Submitting your changes
When submitting CTS or VTS patches in AOSP, choose your development branch based on what API levels the patch applies to.
For changes that apply to multiple API levels, a CL should first be
aosp/masterand then cherry-picked to the most upstream test branch. Allow the automerger to merge the changes downstream in AOSP test branches. See Release schedule and branch information for the list of branches and automerge path information.
- For changes that are specific to a particular API level, develop or CP the changes to the correct test branch with DO NOT MERGE or RESTRICT AUTOMERGE in the commit message.
Follow the Submitting Patches workflow to contribute changes to CTS. A reviewer will be assigned to your change, and your change should be reviewed shortly.
Release schedule and branch information
CTS releases follow this schedule.
|No releases are planned for 6.0, 5.1, 5.0, 4.4, 4.3, and 4.2.|
Important dates during the release
- End of the first week: Code freeze. Submissions to the branch up to the point of code freeze are considered for the upcoming version of CTS. Submissions to the branch after the code freeze, or after a candidate for release is chosen, are considered for the subsequent release.
- Second or third week: CTS is published in the Android Open Source Project (AOSP).
CTS development branches have been set up so that changes submitted to each branch automatically merge to higher branches.
For changes to an upcoming Android version, the automerge path is:
<private-development-branch for Android R>
For changes directly to an AOSP branch, the automerge path is:
If a changelist (CL) fails to merge correctly, the author of the CL is sent an email with instructions on how to resolve the conflict. In most of the cases, the author of the CL can use the instructions to skip the automerge of the conflicting CL.
Additionally, if an older branch requires the change, then the CL needs to be cherry-picked from the newer branch.