There are two options for running CTS automated tests:
Trade Federation is an app and test framework that lets you run tests from the command line. Fundamentally, the framework lets you annotate tests using the
@Test
annotation and the app finds and runs those tests.OmniLab Android Test Station is an app that lets you run tests in a GUI.
This page explains how to run automated tests using Trade Federation. For instructions on running tests using OmniLab Android Test Harness, see OmniLab Android Test Station.
Run all automated CTS tests
Trade Federation contains the CTS console, a command-line interface for running tests. To run the entire automated CTS test suite:
- Ensure you have followed the instructions in Set up CTS automated testing.
On your test workstation, launch the CTS console by running the
cts-tradefed
script from the folder where the CTS package has been unzipped:./android-cts/tools/cts-tradefed
The CTS console displays a cursor for you to type CTS commands (
cts-tradefed >
).The
cts
test plan contains all of the automated CTS tests. Execute the following command to run thects
test plan:run cts
The automated CTS tests execute. Note the following:
Android 13 and higher support multidevice testing. These tests run automatically only when sharding is used. For further information on sharding, see Improve test execution time. If you want to run these tests manually, see Run multidevice tests.
Whenever running CTS, press ctrl+c to stop the existing CTS console and then rerun the CTS console.
You can't run multiple CTS versions on single host because each opens with an existing open location code (OLC) server.
View test results. For more information, see Interpret CTS results.
If this is your first test run, there might be tests that fail due to issues beyond your control. For example, a network connection might be slow or a GPS signal might be weak. Rerun (retry) the tests until all test modules are completed and the test failure numbers are the same between the last two retry sessions:
run retry --retry session_number`
After tests failures are consistent across retries, focus on debugging the remaining failed tests. For information on troubleshooting test results, see Triage test failures and Troubleshoot CTS tests.
Run CTS for multiscreen devices
If your device is running Android 11 or 12, and your device has multiple
screens, you must run the cts-foldable
test plan separately:
run cts-foldable
Passed or failed test cases for alternate screen mode are appended with the
value from display_mode
, for example, testcase1[display_mode=0]
.
Run multidevice tests
Execute the following command to run the cts-multidevice
test plan:
run cts-multidevice
Run individual test plans
Instead of running all test plans at once, you can run individual test plans. This option is useful to save time and to focus on a specific set of tests. To run an individual test plan:
Identify the name of the test plan you want to run by executing the
list modules
command.Run the test plan:
run cts --plan test_module_or_plan_name
Improve test execution time
If you want to improve test execution time, you can shard tests across multiple devices. Sharding requires the host to connect at least two devices, but six or more devices are recommended for efficiency.
To shard tests on Android 11 or higher, run:
run cts --shard-count number_of_shards
Additionally, before running your tests for validation you can improve test
execution time by using the CTS run cts-dev
command. This command skips
preconditions, device-information collection, and all system status checkers.
It also runs the tests on only a single ABI.