Test suite
For a test to be a part of VTS, it must have the following setting
in Android.bp.
test_suites: ["vts"],
Additionally, adding the test to the suite general-tests allows it to be
part of a Test Mapping suite used in presubmit checks.
Test configuration
In most cases, test configuration, which is an XML file used by Trade Federation to run a VTS test, is automatically generated during the build. However, you can customize the test configuration.
Creating a customized test configuration file
Creating a new test XML file from scratch can be complicated, as it involves undestanding how the test harness works, as well as the difference between each test runner. The autogenerated test config file is designed to make this process easier.
If you must customize the test XML file, you can use the autogenerated file as a starting point.
To locate the autogenerated test config file, first
run the make command to build the config, as shown below.
$ m VtsHalUsbV1_1TargetTest
In your build out directory, you can search for the config file based on the module name, as shown below.
$ find out/ -name VtsHalUsbV1_1TargetTest.config
There can be multiple copies of the file and you can use any of them.
Copy the .config file to the directory
where the Android.bp file is.
If there's only one test module in the Android.bp file, you can
rename the XML file to AndroidTest.xml, and the build system automatically
uses that as the test module’s configuration file. Otherwise,
add a test_config attribute to the module, as shown in the
example below.
test_config: "VtsHalUsbV1_1TargetTest.xml",
Now you have a test configuration file to work with and implement the customization.
Forcing the test to run with adb root
Most VTS tests require root privilege to run. If the test configuration
file is autogenerated, you can add following attribute to Android.bp.
require_root: true,
If the test configuration file is customized, add following to the test XML file.
<target_preparer class="com.android.tradefed.targetprep.RootTargetPreparer"/>
Stopping framework during the test
Many VTS tests don't require Android framework to run, and running the
test with framework stopped allows the test to run with stability
without being affected by device flakes. If the test configuration
file is autogenerated, you can add following attribute to Android.bp.
disable_framework: true,
If the test configuration file is customized, add following to the test XML file.
<target_preparer class="com.android.tradefed.targetprep.StopServicesSetup"/>
Adding additional test arguments
Some gtest tests might require more time to run. In such cases, you can add test runner options in the XML file.
For example, the native-test-timeout setting in the following
entry allows the test to run with a timeout of 3 minutes, instead of the
default of 1 minute.
<test class="com.android.tradefed.testtype.GTest" >
<option name="native-test-device-path" value="/data/local/tmp" />
<option name="module-name" value="VtsHalNfcV1_0TargetTest" />
<option name="native-test-timeout" value="180000"/>
</test>
Requiring minimum API level
Some VTS tests can run only on devices with minimum API level. If the
test configuration file is autogenerated, you can add following
attribute to Android.bp.
test_min_api_level: 29,
If the test configuration file is customized, add the following command to the test XML file.
<object type="module_controller" class="com.android.tradefed.testtype.suite.module.MinApiLevelModuleController">
<option name="min-api-level" value="29" />
</object>
Sharding process using VTS
For Android version 10 or higher, you can perform the sharding process on multiple devices while testing with both VTS and CTS-on-GSI plans by following the instructions below.
run vts --shard-count <number of devices> -s <device serial> ...
This command splits the VTS plan into shards and runs them on multiple devices.
run cts-on-gsi --shard-count <number of devices> -s <device serial> -s ...
This command splits the CTS-on-GSI plan into shards and runs them on multiple devices.