Google is committed to advancing racial equity for Black communities. See how.

Test Setup

Test suite

For a test to be a part of VTS, it must have the following setting in Android.bp.

test_suites: ["vts"],

Additionally, adding the test to the suite general-tests allows it to be part of a Test Mapping suite used in presubmit checks.

Test configuration

In most cases, test configuration, which is an XML file used by Trade Federation to run a VTS test, is automatically generated during the build. However, you can customize the test configuration.

Creating a customized test configuration file

Creating a new test XML file from scratch can be complicated, as it involves undestanding how the test harness works, as well as the difference between each test runner. The autogenerated test config file is designed to make this process easier.

If you must customize the test XML file, you can use the autogenerated file as a starting point.

To locate the autogenerated test config file, first run the make command to build the config, as shown below.

$ m VtsHalUsbV1_1TargetTest

In your build out directory, you can search for the config file based on the module name, as shown below.

$ find out/ -name VtsHalUsbV1_1TargetTest.config

There can be multiple copies of the file and you can use any of them. Copy the .config file to the directory where the Android.bp file is.

If there's only one test module in the Android.bp file, you can rename the XML file to AndroidTest.xml, and the build system automatically uses that as the test module’s configuration file. Otherwise, add a test_config attribute to the module, as shown in the example below.

test_config: "VtsHalUsbV1_1TargetTest.xml",

Now you have a test configuration file to work with and implement the customization.

Forcing the test to run with adb root

Most VTS tests require root privilege to run. If the test configuration file is autogenerated, you can add following attribute to Android.bp.

require_root: true,

If the test configuration file is customized, add following to the test XML file.

<target_preparer class="com.android.tradefed.targetprep.RootTargetPreparer"/>

Stopping framework during the test

Many VTS tests don't require Android framework to run, and running the test with framework stopped allows the test to run with stability without being affected by device flakes. If the test configuration file is autogenerated, you can add following attribute to Android.bp.

disable_framework: true,

If the test configuration file is customized, add following to the test XML file.

<target_preparer class="com.android.tradefed.targetprep.StopServicesSetup"/>

Adding additional test arguments

Some gtest tests might require more time to run. In such cases, you can add test runner options in the XML file.

For example, the native-test-timeout setting in the following entry allows the test to run with a timeout of 3 minutes, instead of the default of 1 minute.

   <test class="com.android.tradefed.testtype.GTest" >
       <option name="native-test-device-path" value="/data/local/tmp" />
       <option name="module-name" value="VtsHalNfcV1_0TargetTest" />
       <option name="native-test-timeout" value="180000"/>
   </test>

Requiring minimum API level

Some VTS tests can run only on devices with minimum API level. If the test configuration file is autogenerated, you can add following attribute to Android.bp.

test_min_api_level: 29,

If the test configuration file is customized, add the following command to the test XML file.

   <object type="module_controller" class="com.android.tradefed.testtype.suite.module.MinApiLevelModuleController">
       <option name="min-api-level" value="29" />
   </object>

Sharding process using VTS

For Android version 10 or higher, you can perform the sharding process on multiple devices while testing with both VTS and CTS-on-GSI plans by following the instructions below.

run vts --shard-count <number of devices> -s <device serial> ...

This command splits the VTS plan into shards and runs them on multiple devices.

run cts-on-gsi --shard-count <number of devices> -s <device serial> -s ...

This command splits the CTS-on-GSI plan into shards and runs them on multiple devices.