ITestSuite
public
abstract
class
ITestSuite
extends Object
implements
IBuildReceiver,
IConfigurationReceiver,
IDeviceTest,
IInvocationContextReceiver,
IMetricCollectorReceiver,
IRemoteTest,
IReportNotExecuted,
IRuntimeHintProvider,
IShardableTest,
ISystemStatusCheckerReceiver,
ITestCollector,
ITestLoggerReceiver,
ITokenRequest
java.lang.Object | |
↳ | com.android.tradefed.testtype.suite.ITestSuite |
Abstract class used to run Test Suite. This class provide the base of how the Suite will be run.
Each implementation can define the list of tests via the loadTests()
method.
Summary
Fields | |
---|---|
public
static
final
String |
ABI_OPTION
|
public
static
final
String |
ACTIVE_MAINLINE_PARAMETER_KEY
|
public
static
final
String |
BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY
|
public
static
final
String |
ENABLE_RESOLVE_SYM_LINKS
|
public
static
final
String |
MAINLINE_PARAMETER_KEY
|
public
static
final
String |
MODULE_CHECKER_POST
|
public
static
final
String |
MODULE_CHECKER_PRE
|
public
static
final
String |
MODULE_END_TIME
|
public
static
final
String |
MODULE_METADATA_EXCLUDE_FILTER
|
public
static
final
String |
MODULE_METADATA_INCLUDE_FILTER
|
public
static
final
String |
MODULE_START_TIME
|
public
static
final
String |
PARAMETER_KEY
|
public
static
final
String |
PREPARER_WHITELIST
|
public
static
final
String |
PRIMARY_ABI_RUN
|
public
static
final
String |
RANDOM_SEED
|
public
static
final
String |
RUNNER_WHITELIST
|
public
static
final
String |
SKIP_HOST_ARCH_CHECK
|
public
static
final
String |
SKIP_STAGING_ARTIFACTS
|
public
static
final
String |
SKIP_SYSTEM_STATUS_CHECKER
|
public
static
final
String |
STAGE_MODULE_ARTIFACTS
|
public
static
final
String |
TEST_TYPE_KEY
|
public
static
final
String |
TEST_TYPE_VALUE_PERFORMANCE
|
public
static
final
String |
TOKEN_KEY
|
protected
boolean |
mEnableResolveSymlinks
|
protected
boolean |
mRecoverDeviceByCvd
|
Public constructors | |
---|---|
ITestSuite()
|
Public methods | |
---|---|
void
|
addModuleMetadataExcludeFilters(MultiMap<String, String> filters)
|
void
|
addModuleMetadataIncludeFilters(MultiMap<String, String> filters)
|
void
|
cleanUpSuiteSetup()
Opportunity to clean up all the things that were needed during the suites setup but are not required to run the tests. |
boolean
|
filterByConfigMetadata(IConfiguration config, MultiMap<String, String> include, MultiMap<String, String> exclude)
Apply the metadata filter to the config and see if the config should run. |
|
getAbis(ITestDevice device)
Gets the set of ABIs supported by both Compatibility testing |
static
|
getAbisForBuildTargetArchFromSuite()
Returns the possible abis from the TestSuiteInfo. |
IBuildInfo
|
getBuildInfo()
Implementation of |
final
IConfiguration
|
getConfiguration()
Returns the invocation |
ITestLogger
|
getCurrentTestLogger()
|
ITestDevice
|
getDevice()
Get the device under test. |
ModuleDefinition
|
getDirectModule()
Returns the |
boolean
|
getIntraModuleSharding()
|
IInvocationContext
|
getInvocationContext()
Returns the invocation context. |
MultiMap<String, String>
|
getModuleMetadataIncludeFilters()
|
ITestSuite.MultiDeviceModuleStrategy
|
getMultiDeviceStrategy()
|
final
String
|
getRequestedAbi()
Returns the abi requested with the option -a or --abi. |
|
getRequiredTokens(TestInformation testInfo)
Returns the list of required tokens by the test. |
long
|
getRuntimeHint()
Return the expected runtime of the test in milliseconds. |
File
|
getTestsDir()
|
boolean
|
isSplitting()
Returns true if we are currently in |
abstract
|
loadTests()
Abstract method to load the tests configuration that will be run. |
void
|
reportNotExecuted(ITestInvocationListener listener, String message)
Report the non-executed tests to the main listener provided. |
void
|
reportNotExecuted(ITestInvocationListener listener)
Report the non-executed tests to the main listener provided. |
final
void
|
run(TestInformation testInfo, ITestInvocationListener listener)
Generic run method for all test loaded from |
void
|
setAbiName(String abiName)
Set the value of mAbiName |
final
void
|
setAbis(
|
void
|
setBuild(IBuildInfo buildInfo)
|
void
|
setCollectTestsOnly(boolean shouldCollectTest)
Run the test suite in collector only mode, this requires all the sub-tests to implements this interface too. |
void
|
setConfiguration(IConfiguration configuration)
Injects the |
void
|
setDevice(ITestDevice device)
Inject the device under test. |
void
|
setDirectModule(ModuleDefinition module)
|
void
|
setIntraModuleSharding(boolean intraModuleSharding)
|
void
|
setInvocationContext(IInvocationContext invocationContext)
|
void
|
setMetricCollectors(
Sets the list of |
void
|
setMultiDeviceStrategy(ITestSuite.MultiDeviceModuleStrategy strategy)
|
void
|
setPrimaryAbiRun(boolean primaryAbiRun)
Set the value of mPrimaryAbiRun |
void
|
setShouldMakeDynamicModule(boolean dynamicModule)
When doing distributed sharding, we cannot have ModuleDefinition that shares tests in a pool otherwise intra-module sharding will not work, so we allow to disable it. |
void
|
setSkipContext(SkipContext skipContext)
|
void
|
setSystemStatusChecker(
Sets the |
void
|
setTestLogger(ITestLogger testLogger)
Injects the |
|
split(Integer shardCountHint, TestInformation testInfo)
Alternative version of |
Protected methods | |
---|---|
|
createModuleListeners()
Returns the list of |
boolean
|
filterByRunnerType(IConfiguration config,
Apply the Runner whitelist filtering, removing any runner that was not whitelisted. |
|
getAbisForBuildTargetArch()
Return the abis supported by the Host build target architecture. |
|
getHostAbis()
Returns the host machine abis. |
void
|
setPrioritizeHostConfig(boolean prioritizeHostConfig)
Set option prioritize-host-config. |
boolean
|
shouldModuleRun(ModuleDefinition module)
|
Fields
ABI_OPTION
public static final String ABI_OPTION
ACTIVE_MAINLINE_PARAMETER_KEY
public static final String ACTIVE_MAINLINE_PARAMETER_KEY
BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY
public static final String BUILD_ATTRIBUTE_FLAG_OVERRIDES_KEY
ENABLE_RESOLVE_SYM_LINKS
public static final String ENABLE_RESOLVE_SYM_LINKS
MAINLINE_PARAMETER_KEY
public static final String MAINLINE_PARAMETER_KEY
MODULE_CHECKER_POST
public static final String MODULE_CHECKER_POST
MODULE_CHECKER_PRE
public static final String MODULE_CHECKER_PRE
MODULE_END_TIME
public static final String MODULE_END_TIME
MODULE_METADATA_EXCLUDE_FILTER
public static final String MODULE_METADATA_EXCLUDE_FILTER
MODULE_METADATA_INCLUDE_FILTER
public static final String MODULE_METADATA_INCLUDE_FILTER
MODULE_START_TIME
public static final String MODULE_START_TIME
PARAMETER_KEY
public static final String PARAMETER_KEY
PREPARER_WHITELIST
public static final String PREPARER_WHITELIST
PRIMARY_ABI_RUN
public static final String PRIMARY_ABI_RUN
RANDOM_SEED
public static final String RANDOM_SEED
RUNNER_WHITELIST
public static final String RUNNER_WHITELIST
SKIP_HOST_ARCH_CHECK
public static final String SKIP_HOST_ARCH_CHECK
SKIP_STAGING_ARTIFACTS
public static final String SKIP_STAGING_ARTIFACTS
SKIP_SYSTEM_STATUS_CHECKER
public static final String SKIP_SYSTEM_STATUS_CHECKER
STAGE_MODULE_ARTIFACTS
public static final String STAGE_MODULE_ARTIFACTS
TEST_TYPE_KEY
public static final String TEST_TYPE_KEY
TEST_TYPE_VALUE_PERFORMANCE
public static final String TEST_TYPE_VALUE_PERFORMANCE
TOKEN_KEY
public static final String TOKEN_KEY
mEnableResolveSymlinks
protected boolean mEnableResolveSymlinks
mRecoverDeviceByCvd
protected boolean mRecoverDeviceByCvd
Public constructors
ITestSuite
public ITestSuite ()
Public methods
addModuleMetadataExcludeFilters
public void addModuleMetadataExcludeFilters (MultiMap<String, String> filters)
Parameters | |
---|---|
filters |
MultiMap |
addModuleMetadataIncludeFilters
public void addModuleMetadataIncludeFilters (MultiMap<String, String> filters)
Parameters | |
---|---|
filters |
MultiMap |
cleanUpSuiteSetup
public void cleanUpSuiteSetup ()
Opportunity to clean up all the things that were needed during the suites setup but are not required to run the tests.
filterByConfigMetadata
public boolean filterByConfigMetadata (IConfiguration config, MultiMap<String, String> include, MultiMap<String, String> exclude)
Apply the metadata filter to the config and see if the config should run.
Parameters | |
---|---|
config |
IConfiguration : The IConfiguration being evaluated. |
include |
MultiMap : the metadata include filter |
exclude |
MultiMap : the metadata exclude filter |
Returns | |
---|---|
boolean |
True if the module should run, false otherwise. |
getAbis
publicgetAbis (ITestDevice device)
Gets the set of ABIs supported by both Compatibility testing AbiUtils.getAbisSupportedByCompatibility()
and the device under test.
Parameters | |
---|---|
device |
ITestDevice |
Returns | |
---|---|
|
The set of ABIs to run the tests on |
Throws | |
---|---|
|
com.android.tradefed.device.DeviceNotAvailableException |
DeviceNotAvailableException |
getAbisForBuildTargetArchFromSuite
public staticgetAbisForBuildTargetArchFromSuite ()
Returns the possible abis from the TestSuiteInfo.
Returns | |
---|---|
|
getBuildInfo
public IBuildInfo getBuildInfo ()
Implementation of ITestSuite
may require the build info to load the tests.
Returns | |
---|---|
IBuildInfo |
getConfiguration
public final IConfiguration getConfiguration ()
Returns the invocation IConfiguration
.
Returns | |
---|---|
IConfiguration |
getDevice
public ITestDevice getDevice ()
Get the device under test.
Returns | |
---|---|
ITestDevice |
the ITestDevice |
getDirectModule
public ModuleDefinition getDirectModule ()
Returns the ModuleDefinition
to be executed directly, or null if none yet (when the
ITestSuite has not been sharded yet).
Returns | |
---|---|
ModuleDefinition |
getIntraModuleSharding
public boolean getIntraModuleSharding ()
Returns | |
---|---|
boolean |
getInvocationContext
public IInvocationContext getInvocationContext ()
Returns the invocation context.
Returns | |
---|---|
IInvocationContext |
getModuleMetadataIncludeFilters
public MultiMap<String, String> getModuleMetadataIncludeFilters ()
Returns | |
---|---|
MultiMap<String, String> |
getMultiDeviceStrategy
public ITestSuite.MultiDeviceModuleStrategy getMultiDeviceStrategy ()
Returns | |
---|---|
ITestSuite.MultiDeviceModuleStrategy |
getRequestedAbi
public final String getRequestedAbi ()
Returns the abi requested with the option -a or --abi.
Returns | |
---|---|
String |
getRequiredTokens
publicgetRequiredTokens (TestInformation testInfo)
Returns the list of required tokens by the test. Returns null if no token support.
Parameters | |
---|---|
testInfo |
TestInformation |
Returns | |
---|---|
|
getRuntimeHint
public long getRuntimeHint ()
Return the expected runtime of the test in milliseconds. The time is used to load balance sharded execution
Returns | |
---|---|
long |
getTestsDir
public File getTestsDir ()
Returns | |
---|---|
File |
isSplitting
public boolean isSplitting ()
Returns true if we are currently in IShardableTest.split(int)
.
Returns | |
---|---|
boolean |
loadTests
public abstractloadTests ()
Abstract method to load the tests configuration that will be run. Each tests is defined by a
IConfiguration
and a unique name under which it will report results.
Returns | |
---|---|
|
reportNotExecuted
public void reportNotExecuted (ITestInvocationListener listener, String message)
Report the non-executed tests to the main listener provided. They should be reported as
failed with the NOT_EXECUTED_FAILURE
message.
Parameters | |
---|---|
listener |
ITestInvocationListener : the main listener where to report the non-executed results. |
message |
String : the message to be associated with the non-executed failure. |
reportNotExecuted
public void reportNotExecuted (ITestInvocationListener listener)
Report the non-executed tests to the main listener provided. They should be reported as
failed with the NOT_EXECUTED_FAILURE
message.
Parameters | |
---|---|
listener |
ITestInvocationListener : the main listener where to report the non-executed results. |
run
public final void run (TestInformation testInfo, ITestInvocationListener listener)
Generic run method for all test loaded from loadTests()
.
Parameters | |
---|---|
testInfo |
TestInformation : The TestInformation object containing useful information to run
tests. |
listener |
ITestInvocationListener : the ITestInvocationListener of test results |
Throws | |
---|---|
DeviceNotAvailableException |
setAbiName
public void setAbiName (String abiName)
Set the value of mAbiName
Parameters | |
---|---|
abiName |
String |
setAbis
public final void setAbis (abis)
Parameters | |
---|---|
abis |
|
setCollectTestsOnly
public void setCollectTestsOnly (boolean shouldCollectTest)
Run the test suite in collector only mode, this requires all the sub-tests to implements this interface too.
setConfiguration
public void setConfiguration (IConfiguration configuration)
Injects the IConfiguration
in use.
Parameters | |
---|---|
configuration |
IConfiguration |
setDevice
public void setDevice (ITestDevice device)
Inject the device under test.
Parameters | |
---|---|
device |
ITestDevice : the ITestDevice to use |
setDirectModule
public void setDirectModule (ModuleDefinition module)
Parameters | |
---|---|
module |
ModuleDefinition |
setIntraModuleSharding
public void setIntraModuleSharding (boolean intraModuleSharding)
Parameters | |
---|---|
intraModuleSharding |
boolean |
setInvocationContext
public void setInvocationContext (IInvocationContext invocationContext)
Parameters | |
---|---|
invocationContext |
IInvocationContext |
setMetricCollectors
public void setMetricCollectors (collectors)
Sets the list of IMetricCollector
s defined for the test run.
Parameters | |
---|---|
collectors |
|
setMultiDeviceStrategy
public void setMultiDeviceStrategy (ITestSuite.MultiDeviceModuleStrategy strategy)
Parameters | |
---|---|
strategy |
ITestSuite.MultiDeviceModuleStrategy |
setPrimaryAbiRun
public void setPrimaryAbiRun (boolean primaryAbiRun)
Set the value of mPrimaryAbiRun
Parameters | |
---|---|
primaryAbiRun |
boolean |
setShouldMakeDynamicModule
public void setShouldMakeDynamicModule (boolean dynamicModule)
When doing distributed sharding, we cannot have ModuleDefinition that shares tests in a pool otherwise intra-module sharding will not work, so we allow to disable it.
Parameters | |
---|---|
dynamicModule |
boolean |
setSkipContext
public void setSkipContext (SkipContext skipContext)
Parameters | |
---|---|
skipContext |
SkipContext |
setSystemStatusChecker
public void setSystemStatusChecker (systemCheckers)
Sets the ISystemStatusChecker
s from the configuration for the test.
Parameters | |
---|---|
systemCheckers |
|
setTestLogger
public void setTestLogger (ITestLogger testLogger)
Injects the ITestLogger
instance
Parameters | |
---|---|
testLogger |
ITestLogger |
split
publicsplit (Integer shardCountHint, TestInformation testInfo)
Alternative version of split(int)
which also provides a TestInformation
which contains early info from the parent creating the shards. It is useful if things like
device or build information need to be accessed during sharding.
Parameters | |
---|---|
shardCountHint |
Integer : the attempted shard count. |
testInfo |
TestInformation : The parent TestInformation |
Returns | |
---|---|
|
a collection of subtests to be executed separately or null if test is
not currently shardable |
Protected methods
createModuleListeners
protectedcreateModuleListeners ()
Returns the list of ITestInvocationListener
applicable to the ModuleListener
level. These listeners will be re-used for each module, they will not be re-instantiated so
they should not assume an internal state.
Returns | |
---|---|
|
filterByRunnerType
protected boolean filterByRunnerType (IConfiguration config,allowedRunners)
Apply the Runner whitelist filtering, removing any runner that was not whitelisted. If a configuration has several runners, some might be removed and the config will still run.
Parameters | |
---|---|
config |
IConfiguration : The IConfiguration being evaluated. |
allowedRunners |
: The current runner whitelist. |
Returns | |
---|---|
boolean |
True if the configuration module is allowed to run, false otherwise. |
getAbisForBuildTargetArch
protectedgetAbisForBuildTargetArch ()
Return the abis supported by the Host build target architecture. Exposed for testing.
Returns | |
---|---|
|
getHostAbis
protectedgetHostAbis ()
Returns the host machine abis.
Returns | |
---|---|
|
setPrioritizeHostConfig
protected void setPrioritizeHostConfig (boolean prioritizeHostConfig)
Set option prioritize-host-config.
Parameters | |
---|---|
prioritizeHostConfig |
boolean : true to prioritize host config, i.e., run host test if possible. |
shouldModuleRun
protected boolean shouldModuleRun (ModuleDefinition module)
Parameters | |
---|---|
module |
ModuleDefinition |
Returns | |
---|---|
boolean |