Multi-device modules

This doc provides step-by-step instructions on how to create multi-device modules and calls out current limitations when known.

The sample

A CTS wifi-aware multi-device module is provided. It sends a message from one device over wifi and verifies the other device receives it.

The source for the module is at packages/modules/Wifi/tests/hostsidetests/multidevices/test/aware/.

We've annotated the example with as much comments as we feel are useful.

Step 1: Create the module folder

It is recommended to create a folder for your multi-devices module in the suite project it belongs to. For example: cts/hostsidetests/multidevices/. We recommend this so all multi-devices modules remain collocated at least at first which will make it easier to discover examples.

All files for this module should be put under their own module folder. For example: wifi_aware.

Step 2: Create the test

This is where you implement your test logic. It is highly dependent on what is being tested.

Create the Mobly test source, like: wifi_aware_test.py.

Step 3: Create the build file: Android.bp

Add an Android.bp file like packages/modules/Wifi/tests/hostsidetests/multidevices/test/Android.bp. Define a python_test_host module, similar to:

python_test_host {
    name: "CtsWifiAwareTestCases",
    main: "wifi_aware_test.py",
    srcs: ["wifi_aware_test.py"],
    test_suites: [
        "cts",
        "general-tests",
    ],
    test_options: {
        unit_test: false,
    },
    data: [
          // Package the snippet with the mobly test
        ":wifi_aware_snippet",
    ],
}

Specify the snippets for the test with the data field, which will be packed with the binary and can be located and installed in the test by ATest or in Continuous execution.

Mobly Bundled Snippets are available in Android at external/mobly-bundled-snippets/.

Optional: Create custom snippets

Some multi-device modules may require custom Mobly snippets. The sample test includes a wifi-aware snippet at packages/modules/Wifi/tests/hostsidetests/multidevices/com.google.snippet.wifi/aware/WifiAwareSnippet.java, which is built with Mobly Snippet Lib, available in Android at: external/mobly-snippet-lib/.

The snippet should be defined with android_test rule in Android.bp like standard instrumentation:

android_test {
    name: "wifi_aware_snippet",
    sdk_version: "current",
    srcs: [
        "CallbackUtils.java",
        "WifiAwareSnippet.java",
    ],
    manifest: "AndroidManifest.xml",
    static_libs: [
        "androidx.test.runner",
        "guava",
        "mobly-snippet-lib",
    ],
}

Step 4: Create the module config: AndroidTest.xml

Add an AndroidTest.xml file like packages/modules/Wifi/tests/hostsidetests/multidevices/test/aware/AndroidTest.xml. In this test configuration, you need to specify two devices for the test, similar to:

<configuration description="Config for CTS Wifi Aware test cases">
    <option name="test-suite-tag" value="cts" />
    <option name="config-descriptor:metadata" key="component" value="wifi" />
    <option name="config-descriptor:metadata" key="parameter" value="not_instant_app" />
    <option name="config-descriptor:metadata" key="parameter" value="not_multi_abi" />
    <option name="config-descriptor:metadata" key="parameter" value="not_secondary_user" />

    <device name="device1">
        <!-- For coverage to work, the APK should not be uninstalled until after coverage is pulled.
             So it's a lot easier to install APKs outside the python code.
        -->
        <target_preparer class="com.android.tradefed.targetprep.suite.SuiteApkInstaller">
            <option name="test-file-name" value="wifi_aware_snippet.apk" />
        </target_preparer>
        <target_preparer class="com.android.tradefed.targetprep.RunCommandTargetPreparer">
            <option name="run-command" value="input keyevent KEYCODE_WAKEUP" />
            <option name="run-command" value="wm dismiss-keyguard" />
        </target_preparer>
        <target_preparer class="com.android.tradefed.targetprep.PythonVirtualenvPreparer">
          <!-- Any python dependencies can be specified and will be installed with pip -->
          <option name="dep-module" value="mobly" />
        </target_preparer>
    </device>
    <device name="device2">
        <target_preparer class="com.android.tradefed.targetprep.suite.SuiteApkInstaller">
            <option name="test-file-name" value="wifi_aware_snippet.apk" />
        </target_preparer>
        <target_preparer class="com.android.tradefed.targetprep.RunCommandTargetPreparer">
            <option name="run-command" value="input keyevent KEYCODE_WAKEUP" />
            <option name="run-command" value="wm dismiss-keyguard" />
        </target_preparer>
    </device>

    <test class="com.android.tradefed.testtype.mobly.MoblyBinaryHostTest">
      <!-- The mobly-par-file-name should match the module name -->
      <option name="mobly-par-file-name" value="CtsWifiAwareTestCases" />
      <!-- Timeout limit in milliseconds for all test cases of the python binary -->
      <option name="mobly-test-timeout" value="60000" />
    </test>
</configuration>

Notce that:

  • This sample test has dependency on Mobly. Any dependency can be specified for PythonVirtualenvPreparer and will be installed with pip.
  • The mobly-par-file-namefor MoblyBinaryHostTest must match the module name as in Android.bp.
  • Do specify a mobly-test-timeout for the test. It is in milliseconds and applies to the complete python binary execution (all test cases together). This is needed to avoid test cases hanging forever in case of some issues.
  • Each device tag can contain a distinct setup on each device, The Mobly config will receive them in the same order as specified in the XML.

Related to snippet apk installation:

  • The initial POC has been updated to install snippet apks via target_preparer due to conversation with Coverage team: In order to ensure coverage measurements are not deleted too early, uninstalling by Harness rather than by test code in Python binaries offer better guarantees in terms of timing.

Step 5: Run test locally: atest

Currently, multi-device tests only runs on physical devices. Before running the test, verify that your test devices are in proper state. The command adb devices should report the list of your connected devices. If the list contains devices not intended for testing, specify the devices for the test using -s flag.

For wifi tests, make sure wifi is enabled for the devices (after factory reset).

You can run the test locally with atest:

$ atest CtsWifiAwareTestCases

You should see the number of devices used in the summary heading in atest output, something like Test executed with 2 device(s).

Troubleshooting

If the test fails when running locally due to:

Virtualenv error

java.io.IOException: Cannot run program
"virtualenv": error=2, No such file or directory

Please ensure virtualenv is in your PATH. Adding "~/.local/bin" to PATH should fix it. if virtualenv isn't installed follow: https://virtualenv.pypa.io/en/latest/installation.html

Expected to get at least 2 controller objects, got 1

Test modules are either multi-devices or single-device, there are no mixed-modules. If you attempt to run a multi-devices module without multiple devices you will see this error:

Expected to get at least 2 controller objects, got 1

Executing the module in multi-devices mode will solve the issue.

For CTS: You can use sharding to trigger it (For example: --shard-count 2) or run cts-multidevces.