AndroidTest.xml structure

The overall structure of the module configuration follows a similar pattern to the regular Tradefed XML configuration but with some restrictions due to the fact that they run as part of a suite.

List of allowed tags

AndroidTest.xml or more broadly module configuration can contain only the following XML tags: target_preparer, multi_target_preparer, test and metrics_collector.

Although that list looks restrictive, it allows you to precisely define test module setup needs and the test to run.

NOTE: See Tradefed XML configuration if you need a refresher on the different tags.

Objects such as build_provider or result_reporter will raise a ConfigurationException if attempted to be run from inside a module configuration. This is meant to avoid the expectation of these objects actually performing some task from within a module.

Example module configuration

<configuration description="Config for CTS Gesture test cases">
    <option name="test-suite-tag" value="cts" />
    <target_preparer class="com.android.tradefed.targetprep.suite.SuiteApkInstaller">
        <option name="cleanup-apks" value="true" />
        <option name="test-file-name" value="CtsGestureTestCases.apk" />
    </target_preparer>
    <test class="com.android.tradefed.testtype.AndroidJUnitTest" >
        <option name="package" value="android.gesture.cts" />
        <option name="runtime-hint" value="10m50s" />
    </test>
</configuration>

This configuration describes a test that requires CtsGestureTestCases.apk to be installed and will run an instrumentation against the android.gesture.cts package.

Inclusion tags <include> and <template-include>

Using of <include> and <template-include> in module configs is discouraged. They are not guaranteed to work as expected.

Special case for metrics_collector tag

The metrics_collector is allowed but limited to the FilePullerLogCollector class in order to specify a given file or directory to be pulled and logged for the module. This is useful if you are leaving logs in a particular location and would like to automatically recover them.

Example configuration:

<configuration description="Config for CTS UI Rendering test cases">
    <target_preparer class="com.android.tradefed.targetprep.suite.SuiteApkInstaller">
        <option name="cleanup-apks" value="true" />
        <option name="test-file-name" value="CtsUiRenderingTestCases.apk" />
    </target_preparer>
    <test class="com.android.tradefed.testtype.AndroidJUnitTest" >
        <option name="package" value="android.uirendering.cts" />
        <option name="runtime-hint" value="11m55s" />
        <option name="runner" value="android.uirendering.cts.runner.UiRenderingRunner" />
        <option name="isolated-storage" value="false" />
    </test>

    <!-- Collect the files in the dump directory for debugging -->
    <metrics_collector class="com.android.tradefed.device.metric.FilePullerLogCollector">
        <option name="directory-keys" value="/sdcard/UiRenderingCaptures" />
        <option name="collect-on-run-ended-only" value="true" />
    </metrics_collector>
</configuration>

What about build infos or downloads?

The definition of the allowed tags might give the incorrect impression that a module will not get any build information. This is not true.

The build information is provided from the suite-level setup and will be shared by all the modules of the suite. This allows a single top-level setup for the suite in order to run all the modules part of the suite.

For example, instead of each Compatibility Test Suite (CTS) module individually querying the device information, types, etc., the CTS suite-level setup (cts.xml) does it once and each module will receive that information if they request it.

In order for the objects in a module to receive the build information, they need to do the same as in regular Tradefed configuration: implement the IBuildReceiver interface to receive the IBuildInfo. See testing with device for more details.

Metadata fields

A large number of test modules include some metadata specifications, which each have a unique goal.

Example:

  <option name="config-descriptor:metadata" key="component" value="framework" />
  <option name="config-descriptor:metadata" key="parameter" value="instant_app" />
  <option name="config-descriptor:metadata" key="parameter" value="multi_abi" />
  <option name="config-descriptor:metadata" key="parameter" value="secondary_user" />

Component

The component metadata describes the general Android component the module intends to test. It doesn't have any direct impact on the test execution; it's primarily used for organizational purposes.

The up-to-date list of allowed components for CTS is available in CtsConfigLoadingTest. This test fails in presubmit if a non-existing component is added to a CTS module.

You can filter a suite run based on the components using module-metadata-include-filter and module-metadata-exclude-filter.

Example:

  --module-metadata-include-filter component framework

This example only runs the test module annotated with the framework component.

Parameter

The parameter metadata is informational and impacts the test execution. It specifies which Android mode the test module applies to. In this case, modes are limited to high level Android modes, such as instant apps, secondary users or different abis.

During the suite run, if the mode applies to the test, several variation of the test module are created based on the mode. Each variation runs similar tests but under different modes.

  • instant_app: Create a variation of the tests that install APKs as instant apps.
  • multi_abi: Create a variation of the tests for each ABI supported by the device.
  • secondary_user: Create a variation of the tests that install APKs and run tests as a secondary user.

Metric collecting and post-processing for performance test modules

For performance test modules, module-level metrics_collector and metric_post_processor are allowed as they are essential to performance tests. The module-level metric collectors and post-processors can be module specific. It is not recommended to specify post-processors at both top-level and module-level.

A performance test module configuration must include the test-type metadata with value performance, like: xml <option name="config-descriptor:metadata" key="test-type" value="performance" /> Without this, if a test config includes metric_collector other than FilePullerLogCollector or any metric_post_processor, the test fails in presubmit.

Example performance test module configuration:

<configuration description="Runs sample performance test.">
    <!-- Declare as a performance test module -->
    <option name="config-descriptor:metadata" key="test-type" value="performance" />
    <option name="test-tag" value="hello-world-performance-test" />
    <test class="com.android.tradefed.testtype.HostTest" >
        <option name="class" value="android.test.example.helloworldperformance.HelloWorldPerformanceTest" />
    </test>
    <!-- Add module-level post processor MetricFilePostProcessor -->
    <metric_post_processor class="com.android.tradefed.postprocessor.MetricFilePostProcessor">
        <option name="aggregate-similar-tests" value="true" />
        <option name="enable-per-test-log" value="false" />
    </metric_post_processor>
</configuration>