Complete Automotive Tests in a Box

Stay organized with collections Save and categorize content based on your preferences.

Complete Automotive Tests in a Box (CATBox) is an open source package that provides the frameworks and tools required to streamline and execute automotive tests with minimal configuration. CATBox is scalable and compatible with the tools, infrastructure, and tests you use to test and validate your Android Automotive OS (AAOS) implementations. As a result, you can provide high-quality products to the marketplace.

CATBox design and implementation

The CATBox test suite provides the tools, frameworks, and tests needed to execute functional and performance tests on an automotive target device and while using a companion device.

The following image illustrates a high level design of a CATBox package.

Figure 1. CATBox package

Tradefed

Tradefed is an open source continuous testing framework used to execute tests on Android devices. For details, see Trade Federation Overview.

Automotive test framework

CATBox is based on an Automotive test framework called Spectatio, which provides APIs for testing various apps on automotive devices. It is built on top of UI Automator, an open source testing framework, that provides a set of APIs to build UI tests that interact with user and system applications.

CATBox test suite downloads

Android 14

Android Automotive 14 is the release of the development milestone code-named U. Use the following links to download CATBox test suites for Android 14:

• Android 14 (arm64)
• Android 14 (x86_64)

Android 12

Android 12 is the release of the development milestone code-named S. Use the following links to download CATBox test suites for Android 12:

• Android 12 (arm64)
• Android 12 (x86_64)

Android 11

Android 11 is the release of the development milestone code-named R. Use the following links to download CATBox test suites for Android 11:

• Android 11 (arm64)
• Android 11 (x86_64)

Build CATBox

Use the catbox build command once the Android AOSP code is available on the local workstation.

To build the CATBox package locally:

1. Select the Device Target based on the device architecture and run the following command:

   lunch <target>

2. To build CATBox, run the following command:

   make catbox

   This command creates the android-catbox.zip file in the /out/host/linux-x86/catbox directory, which can then be used to run tests.

Set up the environment

The environment setup required to execute CATBox tests is similar to Setting up CTS. To set up the environment, complete each of the following tasks:

• Install ADB and AAPT.

• Install JDK. See Java Development Kit for Ubuntu.

• Increase memory limit. See Storage requirements.

Set up and run CATBox

Use the Tradefed test framework to run the tests on Android devices. Before you set up CATBox, see the Trade Federation Overview.

To set up and run CATBox, see the sections below.

Set up the device

Before running a test plan, complete the following steps to set up your device:

1. Ensure that you have installed the recent versions of both the Android Debug Bridge (adb) and the Android Asset Packaging Tool (AAPT), as well as added the location of these tools to the system path of your machine. For details, see ADB and AAPT.

2. Connect at least one device and prepare the Device Under Test (DUT) as follows:

   1. To factory reset the automotive device, go to Settings > System > Reset Options > Erase all data.

   2. Wait for the device to automatically restart.

   3. Make sure adb is enabled. To confirm the device is available, run the following command:

      adb devices

   4. To enable root, run the following command:

      adb -s DEVICE_SERIAL root

      Replace DEVICE_SERIAL with the Serial ID for the automotive device.

   5. Connect to Wi-Fi.

Extract the CATBox package

After setting up the device, unzip the CATBox package and go to the extracted folder:

unzip android-catbox.zip
cd android-catbox

Execute tests in CATBox

To run CATBox with the test plans in the CATBox package:

./tools/catbox-tradefed run commandAndExit TEST_PLAN_NAME --serial DEVICE_SERIAL

Replace DEVICE_SERIAL with the Serial ID for the automotive device. If you have multiple devices connected to the host machine, use DEVICE_SERIAL to distinguish them. Replace TEST_PLAN_NAME with the name of the test plan you want to run.

To get the most complete list of available plans, use the command line:

./tools/catbox-tradefed list plans | grep -i catbox

Types of test plans

The following table displays the available functional test plans.

The following table lists CATBox performance test plans.

Run the multidevice host side test

The execution of multidevice host side tests uses CATBox to run the test in one session. For example, BTDiscoveryTest:

make catbox
./tools/catbox-tradefed run commandAndExit catbox-functional-bluetooth-tests --{phone}serial <serial> --{auto}serial <serial> --mobly-host:mobly-par-file-name BTDiscoveryTest

CATBox installs the apk and then the test is run in the correct Python execution environment.

Review CATBox test results

Results of functional and performance test runs are saved in android-catbox/results/latest.