Structure of a test runner

The test runner is the execution unit of the invocation flow. This is where tests actually run.

Interfaces

Test runners are defined via the IRemoteTest interface, which provides a simple run method to implement that will be called when the tests is to run.

This allows the simplest definition of a test run to occur. But in practice, test writers will need more information to properly write their tests, typically build and device information. This is where the following interfaces come handy.

Basic

These two interfaces are the most widely used today, as they represent the basic needs of most tests.

  • IBuildReceiver allows the test to get the IBuildInfo object created at the build provider step containing all the information and artifacts related to the test setup.
  • IDeviceTest allows TF to receive the ITestDevice object that represents the device under test and provides an API to interact with it.

Advanced

There are additional interfaces that allow more complex interaction between the test harness and the test runner:

  • ITestFilterReceiver, which allows the test to receive a set of filters for running certain tests only. This is useful in running a subset of the tests.
  • ITestCollector, which allows a test runner to only dry-run the tests instead of actually executing them. This is useful in collecting the list of all test cases.

Existing test runners

A variety of test runners already exists, some for major test types:

A large number of custom test runners exist besides the above; they serve specialized purposes for some functional testing, for example Boot Test.

Write a new test runner

More guidance of writing a new test runner is available in the writing tests section.