ITestSuite

public abstract class ITestSuite
extends Object implements IBuildReceiver, IConfigurationReceiver, IDeviceTest, IInvocationContextReceiver, IMetricCollectorReceiver, IRemoteTest, IReportNotExecuted, IRuntimeHintProvider, IShardableTest, ISystemStatusCheckerReceiver, ITestCollector, ITestLoggerReceiver, ITokenRequest

java.lang.Object
   ↳ com.android.tradefed.testtype.suite.ITestSuite


Abstract class used to run Test Suite. This class provide the base of how the Suite will be run. Each implementation can define the list of tests via the loadTests() method.

Summary

Fields

public static final String ABI_OPTION

public static final String ACTIVE_MAINLINE_PARAMETER_KEY

public static final String BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY

public static final String ENABLE_RESOLVE_SYM_LINKS

public static final String MAINLINE_PARAMETER_KEY

public static final String MODULE_CHECKER_POST

public static final String MODULE_CHECKER_PRE

public static final String MODULE_END_TIME

public static final String MODULE_METADATA_EXCLUDE_FILTER

public static final String MODULE_METADATA_INCLUDE_FILTER

public static final String MODULE_START_TIME

public static final String PARAMETER_KEY

public static final String PREPARER_WHITELIST

public static final String PRIMARY_ABI_RUN

public static final String RANDOM_SEED

public static final String RUNNER_WHITELIST

public static final String SKIP_HOST_ARCH_CHECK

public static final String SKIP_STAGING_ARTIFACTS

public static final String SKIP_SYSTEM_STATUS_CHECKER

public static final String STAGE_MODULE_ARTIFACTS

public static final String TEST_TYPE_KEY

public static final String TEST_TYPE_VALUE_PERFORMANCE

public static final String TOKEN_KEY

protected boolean mEnableResolveSymlinks

protected boolean mRecoverDeviceByCvd

Public constructors

ITestSuite()

Public methods

void addModuleMetadataExcludeFilters(MultiMap<String, String> filters)
void addModuleMetadataIncludeFilters(MultiMap<String, String> filters)
void cleanUpSuiteSetup()

Opportunity to clean up all the things that were needed during the suites setup but are not required to run the tests.

boolean filterByConfigMetadata(IConfiguration config, MultiMap<String, String> include, MultiMap<String, String> exclude)

Apply the metadata filter to the config and see if the config should run.

getAbis(ITestDevice device)

Gets the set of ABIs supported by both Compatibility testing AbiUtils.getAbisSupportedByCompatibility() and the device under test.

static getAbisForBuildTargetArchFromSuite()

Returns the possible abis from the TestSuiteInfo.

IBuildInfo getBuildInfo()

Implementation of ITestSuite may require the build info to load the tests.

final IConfiguration getConfiguration()

Returns the invocation IConfiguration.

ITestLogger getCurrentTestLogger()
ITestDevice getDevice()

Get the device under test.

ModuleDefinition getDirectModule()

Returns the ModuleDefinition to be executed directly, or null if none yet (when the ITestSuite has not been sharded yet).

boolean getIntraModuleSharding()
IInvocationContext getInvocationContext()

Returns the invocation context.

MultiMap<String, String> getModuleMetadataIncludeFilters()
ITestSuite.MultiDeviceModuleStrategy getMultiDeviceStrategy()
final String getRequestedAbi()

Returns the abi requested with the option -a or --abi.

getRequiredTokens(TestInformation testInfo)

Returns the list of required tokens by the test.

long getRuntimeHint()

Return the expected runtime of the test in milliseconds.

File getTestsDir()
boolean isSplitting()

Returns true if we are currently in IShardableTest.split(int).

abstract loadTests()

Abstract method to load the tests configuration that will be run.

void reportNotExecuted(ITestInvocationListener listener, String message)

Report the non-executed tests to the main listener provided.

void reportNotExecuted(ITestInvocationListener listener)

Report the non-executed tests to the main listener provided.

final void run(TestInformation testInfo, ITestInvocationListener listener)

Generic run method for all test loaded from loadTests().

void setAbiName(String abiName)

Set the value of mAbiName

final void setAbis( abis)
void setBuild(IBuildInfo buildInfo)

void setCollectTestsOnly(boolean shouldCollectTest)

Run the test suite in collector only mode, this requires all the sub-tests to implements this interface too.

void setConfiguration(IConfiguration configuration)

Injects the IConfiguration in use.

void setDevice(ITestDevice device)

Inject the device under test.

void setDirectModule(ModuleDefinition module)
void setIntraModuleSharding(boolean intraModuleSharding)
void setInvocationContext(IInvocationContext invocationContext)

void setMetricCollectors( collectors)

Sets the list of IMetricCollectors defined for the test run.

void setMultiDeviceStrategy(ITestSuite.MultiDeviceModuleStrategy strategy)
void setPrimaryAbiRun(boolean primaryAbiRun)

Set the value of mPrimaryAbiRun

void setShouldMakeDynamicModule(boolean dynamicModule)

When doing distributed sharding, we cannot have ModuleDefinition that shares tests in a pool otherwise intra-module sharding will not work, so we allow to disable it.

void setSkipContext(SkipContext skipContext)
void setSystemStatusChecker( systemCheckers)

Sets the ISystemStatusCheckers from the configuration for the test.

void setTestLogger(ITestLogger testLogger)

Injects the ITestLogger instance

split(Integer shardCountHint, TestInformation testInfo)

Alternative version of split(int) which also provides a TestInformation which contains early info from the parent creating the shards.

Protected methods

createModuleListeners()

Returns the list of ITestInvocationListener applicable to the ModuleListener level.

boolean filterByRunnerType(IConfiguration config, allowedRunners)

Apply the Runner whitelist filtering, removing any runner that was not whitelisted.

getAbisForBuildTargetArch()

Return the abis supported by the Host build target architecture.

getHostAbis()

Returns the host machine abis.

void setPrioritizeHostConfig(boolean prioritizeHostConfig)

Set option prioritize-host-config.

boolean shouldModuleRun(ModuleDefinition module)

Fields

ABI_OPTION

public static final String ABI_OPTION

ACTIVE_MAINLINE_PARAMETER_KEY

public static final String ACTIVE_MAINLINE_PARAMETER_KEY

BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY

public static final String BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY

public static final String ENABLE_RESOLVE_SYM_LINKS

MAINLINE_PARAMETER_KEY

public static final String MAINLINE_PARAMETER_KEY

MODULE_CHECKER_POST

public static final String MODULE_CHECKER_POST

MODULE_CHECKER_PRE

public static final String MODULE_CHECKER_PRE

MODULE_END_TIME

public static final String MODULE_END_TIME

MODULE_METADATA_EXCLUDE_FILTER

public static final String MODULE_METADATA_EXCLUDE_FILTER

MODULE_METADATA_INCLUDE_FILTER

public static final String MODULE_METADATA_INCLUDE_FILTER

MODULE_START_TIME

public static final String MODULE_START_TIME

PARAMETER_KEY

public static final String PARAMETER_KEY

PREPARER_WHITELIST

public static final String PREPARER_WHITELIST

PRIMARY_ABI_RUN

public static final String PRIMARY_ABI_RUN

RANDOM_SEED

public static final String RANDOM_SEED

RUNNER_WHITELIST

public static final String RUNNER_WHITELIST

SKIP_HOST_ARCH_CHECK

public static final String SKIP_HOST_ARCH_CHECK

SKIP_STAGING_ARTIFACTS

public static final String SKIP_STAGING_ARTIFACTS

SKIP_SYSTEM_STATUS_CHECKER

public static final String SKIP_SYSTEM_STATUS_CHECKER

STAGE_MODULE_ARTIFACTS

public static final String STAGE_MODULE_ARTIFACTS

TEST_TYPE_KEY

public static final String TEST_TYPE_KEY

TEST_TYPE_VALUE_PERFORMANCE

public static final String TEST_TYPE_VALUE_PERFORMANCE

TOKEN_KEY

public static final String TOKEN_KEY

protected boolean mEnableResolveSymlinks

mRecoverDeviceByCvd

protected boolean mRecoverDeviceByCvd

Public constructors

ITestSuite

public ITestSuite ()

Public methods

addModuleMetadataExcludeFilters

public void addModuleMetadataExcludeFilters (MultiMap<String, String> filters)

Parameters
filters MultiMap

addModuleMetadataIncludeFilters

public void addModuleMetadataIncludeFilters (MultiMap<String, String> filters)

Parameters
filters MultiMap

cleanUpSuiteSetup

public void cleanUpSuiteSetup ()

Opportunity to clean up all the things that were needed during the suites setup but are not required to run the tests.

filterByConfigMetadata

public boolean filterByConfigMetadata (IConfiguration config, 
                MultiMap<String, String> include, 
                MultiMap<String, String> exclude)

Apply the metadata filter to the config and see if the config should run.

Parameters
config IConfiguration: The IConfiguration being evaluated.

include MultiMap: the metadata include filter

exclude MultiMap: the metadata exclude filter

Returns
boolean True if the module should run, false otherwise.

getAbis

public  getAbis (ITestDevice device)

Gets the set of ABIs supported by both Compatibility testing AbiUtils.getAbisSupportedByCompatibility() and the device under test.

Parameters
device ITestDevice

Returns
The set of ABIs to run the tests on

Throws
com.android.tradefed.device.DeviceNotAvailableException
DeviceNotAvailableException

getAbisForBuildTargetArchFromSuite

public static  getAbisForBuildTargetArchFromSuite ()

Returns the possible abis from the TestSuiteInfo.

Returns

getBuildInfo

public IBuildInfo getBuildInfo ()

Implementation of ITestSuite may require the build info to load the tests.

Returns
IBuildInfo

getConfiguration

public final IConfiguration getConfiguration ()

Returns the invocation IConfiguration.

Returns
IConfiguration

getCurrentTestLogger

public ITestLogger getCurrentTestLogger ()

Returns
ITestLogger

getDevice

public ITestDevice getDevice ()

Get the device under test.

Returns
ITestDevice the ITestDevice

getDirectModule

public ModuleDefinition getDirectModule ()

Returns the ModuleDefinition to be executed directly, or null if none yet (when the ITestSuite has not been sharded yet).

Returns
ModuleDefinition

getIntraModuleSharding

public boolean getIntraModuleSharding ()

Returns
boolean

getInvocationContext

public IInvocationContext getInvocationContext ()

Returns the invocation context.

Returns
IInvocationContext

getModuleMetadataIncludeFilters

public MultiMap<String, String> getModuleMetadataIncludeFilters ()

Returns
MultiMap<String, String>

getMultiDeviceStrategy

public ITestSuite.MultiDeviceModuleStrategy getMultiDeviceStrategy ()

Returns
ITestSuite.MultiDeviceModuleStrategy

getRequestedAbi

public final String getRequestedAbi ()

Returns the abi requested with the option -a or --abi.

Returns
String

getRequiredTokens

public  getRequiredTokens (TestInformation testInfo)

Returns the list of required tokens by the test. Returns null if no token support.

Parameters
testInfo TestInformation

Returns

getRuntimeHint

public long getRuntimeHint ()

Return the expected runtime of the test in milliseconds. The time is used to load balance sharded execution

Returns
long

getTestsDir

public File getTestsDir ()

Returns
File

isSplitting

public boolean isSplitting ()

Returns true if we are currently in IShardableTest.split(int).

Returns
boolean

loadTests

public abstract  loadTests ()

Abstract method to load the tests configuration that will be run. Each tests is defined by a IConfiguration and a unique name under which it will report results.

Returns

reportNotExecuted

public void reportNotExecuted (ITestInvocationListener listener, 
                String message)

Report the non-executed tests to the main listener provided. They should be reported as failed with the NOT_EXECUTED_FAILURE message.

Parameters
listener ITestInvocationListener: the main listener where to report the non-executed results.

message String: the message to be associated with the non-executed failure.

reportNotExecuted

public void reportNotExecuted (ITestInvocationListener listener)

Report the non-executed tests to the main listener provided. They should be reported as failed with the NOT_EXECUTED_FAILURE message.

Parameters
listener ITestInvocationListener: the main listener where to report the non-executed results.

run

public final void run (TestInformation testInfo, 
                ITestInvocationListener listener)

Generic run method for all test loaded from loadTests().

Parameters
testInfo TestInformation: The TestInformation object containing useful information to run tests.

listener ITestInvocationListener: the ITestInvocationListener of test results

Throws
DeviceNotAvailableException

setAbiName

public void setAbiName (String abiName)

Set the value of mAbiName

Parameters
abiName String

setAbis

public final void setAbis ( abis)

Parameters
abis

setBuild

public void setBuild (IBuildInfo buildInfo)

Parameters
buildInfo IBuildInfo

setCollectTestsOnly

public void setCollectTestsOnly (boolean shouldCollectTest)

Run the test suite in collector only mode, this requires all the sub-tests to implements this interface too.

setConfiguration

public void setConfiguration (IConfiguration configuration)

Injects the IConfiguration in use.

Parameters
configuration IConfiguration

setDevice

public void setDevice (ITestDevice device)

Inject the device under test.

Parameters
device ITestDevice: the ITestDevice to use

setDirectModule

public void setDirectModule (ModuleDefinition module)

Parameters
module ModuleDefinition

setIntraModuleSharding

public void setIntraModuleSharding (boolean intraModuleSharding)

Parameters
intraModuleSharding boolean

setInvocationContext

public void setInvocationContext (IInvocationContext invocationContext)

Parameters
invocationContext IInvocationContext

setMetricCollectors

public void setMetricCollectors ( collectors)

Sets the list of IMetricCollectors defined for the test run.

Parameters
collectors

setMultiDeviceStrategy

public void setMultiDeviceStrategy (ITestSuite.MultiDeviceModuleStrategy strategy)

Parameters
strategy ITestSuite.MultiDeviceModuleStrategy

setPrimaryAbiRun

public void setPrimaryAbiRun (boolean primaryAbiRun)

Set the value of mPrimaryAbiRun

Parameters
primaryAbiRun boolean

setShouldMakeDynamicModule

public void setShouldMakeDynamicModule (boolean dynamicModule)

When doing distributed sharding, we cannot have ModuleDefinition that shares tests in a pool otherwise intra-module sharding will not work, so we allow to disable it.

Parameters
dynamicModule boolean

setSkipContext

public void setSkipContext (SkipContext skipContext)

Parameters
skipContext SkipContext

setSystemStatusChecker

public void setSystemStatusChecker ( systemCheckers)

Sets the ISystemStatusCheckers from the configuration for the test.

Parameters
systemCheckers

setTestLogger

public void setTestLogger (ITestLogger testLogger)

Injects the ITestLogger instance

Parameters
testLogger ITestLogger

split

public  split (Integer shardCountHint, 
                TestInformation testInfo)

Alternative version of split(int) which also provides a TestInformation which contains early info from the parent creating the shards. It is useful if things like device or build information need to be accessed during sharding.

Parameters
shardCountHint Integer: the attempted shard count.

testInfo TestInformation: The parent TestInformation

Returns
a collection of subtests to be executed separately or null if test is not currently shardable

Protected methods

createModuleListeners

protected  createModuleListeners ()

Returns the list of ITestInvocationListener applicable to the ModuleListener level. These listeners will be re-used for each module, they will not be re-instantiated so they should not assume an internal state.

Returns

filterByRunnerType

protected boolean filterByRunnerType (IConfiguration config, 
                 allowedRunners)

Apply the Runner whitelist filtering, removing any runner that was not whitelisted. If a configuration has several runners, some might be removed and the config will still run.

Parameters
config IConfiguration: The IConfiguration being evaluated.

allowedRunners : The current runner whitelist.

Returns
boolean True if the configuration module is allowed to run, false otherwise.

getAbisForBuildTargetArch

protected  getAbisForBuildTargetArch ()

Return the abis supported by the Host build target architecture. Exposed for testing.

Returns

getHostAbis

protected  getHostAbis ()

Returns the host machine abis.

Returns

setPrioritizeHostConfig

protected void setPrioritizeHostConfig (boolean prioritizeHostConfig)

Set option prioritize-host-config.

Parameters
prioritizeHostConfig boolean: true to prioritize host config, i.e., run host test if possible.

shouldModuleRun

protected boolean shouldModuleRun (ModuleDefinition module)

Parameters
module ModuleDefinition

Returns
boolean