Device-side metric collection

When running device-side tests (instrumentations, UI Automator tests, etc.), host-side collectors might not be ideal because it is difficult to synchronize metric collection to a test running on a device. For example, a screenshot taken asynchronously will most likely miss the wanted screen and be useless.

In order to meet these use cases, a device-side version of our collectors exists and can be used in any 'AndroidJUnitRunner' instrumentation. BaseMetricListener can be implemented in order to automatically report metrics that are collected in a way fully compatible with the Tradefed reporting pipeline.

This library is decoupled from Tradefed itself and can be used without Tradefed.

If you are using the 'AndroidJUnitTest' runner from Tradefed, you can simply specify the following command line option to have your collector running with your tests:

  --device-listeners android.device.collectors.ScreenshotListener
  LOCAL_STATIC_JAVA_LIBRARIES += collector-device-lib

Implementation

When implementing on top of the base class BaseMetricListener, you may choose when you would like to collect your metrics during the lifecycle of the instrumentation:

  • When a test run starts: onTestRunStart
  • When a test case starts: onTestStart
  • When a test case ends: onTestEnd
  • When a test case fails: onTestFail
  • When a test run ends: onTestRunEnd

Interaction

The collection of metrics on the device side is made synchronously to the instrumentation execution itself, and metrics are passed back to the instrumentation results and parsed by Tradefed to be reported as part of the invocation.