Run automated tests (Android 11 or higher)

There are two options for running CTS automated tests:

• Trade Federation is an app and test framework that lets you run tests from the command line. Fundamentally, the framework lets you annotate tests using the @Test annotation and the app finds and runs those tests.

• OmniLab Android Test Station is an app that lets you run tests in a GUI.

This page explains how to run automated tests using Trade Federation. For instructions on running tests using OmniLab Android Test Harness, see OmniLab Android Test Station.

Run all automated CTS tests

Trade Federation contains the CTS console, a command-line interface for running tests. To run the entire automated CTS test suite:

1. Ensure you have followed the instructions in Set up CTS automated testing.

2. On your test workstation, launch the CTS console by running the cts-tradefed script from the folder where the CTS package has been unzipped:

   ./android-cts/tools/cts-tradefed
   

   The CTS console displays a cursor for you to type CTS commands (cts-tradefed >).

3. The cts test plan contains all of the automated CTS tests. Execute the following command to run the cts test plan:

   run cts
   

   The automated CTS tests execute. Note the following:

   • Android 13 and higher support multidevice testing. These tests run automatically only when sharding is used. For further information on sharding, see Improve test execution time. If you want to run these tests manually, see Run multidevice tests.

   • Whenever running CTS, press ctrl+c to stop the existing CTS console and then rerun the CTS console.

   • You can't run multiple CTS versions on single host because each opens with an existing open location code (OLC) server.

4. View test results. For more information, see Interpret CTS results.

5. If this is your first test run, there might be tests that fail due to issues beyond your control. For example, a network connection might be slow or a GPS signal might be weak. Rerun (retry) the tests until all test modules are completed and the test failure numbers are the same between the last two retry sessions:

   run retry --retry session_number`
   

6. After tests failures are consistent across retries, focus on debugging the remaining failed tests. For information on troubleshooting test results, see Triage test failures and Troubleshoot CTS tests.

Run CTS for multiscreen devices

If your device is running Android 11 or 12, and your device has multiple screens, you must run the cts-foldable test plan separately:

run cts-foldable

Passed or failed test cases for alternate screen mode are appended with the value from display_mode, for example, testcase1[display_mode=0].

Run multidevice tests (Android 15 or lower)

Execute the following command to run the cts-multidevice test plan:

run cts-multidevice

Run individual test plans

Instead of running all test plans at once, you can run individual test plans. This option is useful to save time and to focus on a specific set of tests. To run an individual test plan:

1. Identify the name of the test plan you want to run by executing the list modules command.

2. Run the test plan:

   run cts --plan test_module_or_plan_name
   

Improve test execution time

If you want to improve test execution time, you can shard tests across multiple devices. Sharding requires the host to connect at least two devices, but six or more devices are recommended for efficiency.

To shard tests on Android 11 or higher, run:

run cts --shard-count number_of_shards

Additionally, before running your tests for validation you can improve test execution time by using the CTS run cts-dev command. This command skips preconditions, device-information collection, and all system status checkers. It also runs the tests on only a single ABI.