CollectingTestListener

public class CollectingTestListener
extends Object implements ITestInvocationListener, ILogSaverListener

java.lang.Object
   ↳ com.android.tradefed.result.CollectingTestListener


A ITestInvocationListener that will collect all test results.

Although the data structures used in this object are thread-safe, the ITestInvocationListener callbacks must be called in the correct order.

Summary

Public constructors

CollectingTestListener()

Public methods

IBuildInfo getBuildInfo()

This method is deprecated. rely on the IBuildInfo from getInvocationContext().

TestRunResult getCurrentRunResults()

Gets the results for the current test run.

IInvocationContext getInvocationContext()

Return the invocation context that was reported via invocationStarted(IInvocationContext)

IInvocationContext getModuleContextForRunResult(TestRunResult res)

Returns the IInvocationContext of the module associated with the results or null if it was not associated with any module.

int getNumAllFailedTestRuns()

Return total number of test runs in a failure state

int getNumAllFailedTests()

Return total number of tests in a failure state (only failed, assumption failures do not count toward it).

int getNumTestsInState(TestResult.TestStatus status)

Gets the number of tests in given state for this run.

int getNumTotalTests()

Gets the total number of complete tests for all runs.

IBuildInfo getPrimaryBuildInfo()

Return the primary build info that was reported via invocationStarted(IInvocationContext).

Collection<TestRunResult> getRunResults()

Gets the results for all test runs.

TestSummary getSummary()

Allows the InvocationListener to return a summary.

boolean hasFailedTests()
boolean hasResultFor(String runName)

Returns True if the result map already has an entry for the run name.

void invocationEnded(long elapsedTime)

Reports that the invocation has terminated, whether successfully or due to some error condition.

void invocationFailed(Throwable cause)

Reports an incomplete invocation due to some error condition.

void invocationStarted(IInvocationContext context)

Reports the start of the test invocation.

void logAssociation(String dataName, LogFile logFile)

In some cases, log must be strongly associated with a test cases, but the opportunity to do so on the direct testLogSaved(String, LogDataType, InputStreamSource, LogFile) callback is not possible.

void setBuildInfo(IBuildInfo buildInfo)

This method is deprecated. Not necessary for testing anymore.

void setLogSaver(ILogSaver logSaver)

Set the ILogSaver to allow the implementor to save files.

void testAssumptionFailure(TestDescription test, String trace)

Called when an atomic test flags that it assumes a condition that is false

void testEnded(TestDescription test, HashMap<String, MetricMeasurement.Metric> testMetrics)

Reports the execution end of an individual test case.

void testEnded(TestDescription test, long endTime, HashMap<String, MetricMeasurement.Metric> testMetrics)

Alternative to testEnded(TestDescription, Map) where we can specify the end time directly.

void testFailed(TestDescription test, String trace)

Reports the failure of a individual test case.

void testIgnored(TestDescription test)

Called when a test will not be run, generally because a test method is annotated with org.junit.Ignore.

void testLog(String dataName, LogDataType dataType, InputStreamSource dataStream)

Provides the associated log or debug data from the test invocation.

void testLogSaved(String dataName, LogDataType dataType, InputStreamSource dataStream, LogFile logFile)

Called when the test log is saved.

void testModuleEnded()

Reports the end of a module run.

void testModuleStarted(IInvocationContext moduleContext)

Reports the beginning of a module running.

void testRunEnded(long elapsedTime, HashMap<String, MetricMeasurement.Metric> runMetrics)

Reports end of test run.

void testRunFailed(String errorMessage)

Reports test run failed to complete due to a fatal error.

void testRunStarted(String name, int numTests)

Reports the start of a test run.

void testRunStopped(long elapsedTime)

Reports test run stopped before completion due to a user request.

void testStarted(TestDescription test, long startTime)

Alternative to testStarted(TestDescription) where we also specify when the test was started, combined with testEnded(TestDescription, long, Map) for accurate measure.

void testStarted(TestDescription test)

Reports the start of an individual test case.

Protected methods

void setIsAggregrateMetrics(boolean aggregate)

Toggle the 'aggregate metrics' option

Public constructors

CollectingTestListener

public CollectingTestListener ()

Public methods

getBuildInfo

public IBuildInfo getBuildInfo ()

This method is deprecated.
rely on the IBuildInfo from getInvocationContext().

Returns the build info.

Returns
IBuildInfo

getCurrentRunResults

public TestRunResult getCurrentRunResults ()

Gets the results for the current test run.

Note the results may not be complete. It is recommended to test the value of TestRunResult.isRunComplete() and/or (@link TestRunResult#isRunFailure()} as appropriate before processing the results.

Returns
TestRunResult the TestRunResult representing data collected during last test run

getInvocationContext

public IInvocationContext getInvocationContext ()

Return the invocation context that was reported via invocationStarted(IInvocationContext)

Returns
IInvocationContext

getModuleContextForRunResult

public IInvocationContext getModuleContextForRunResult (TestRunResult res)

Returns the IInvocationContext of the module associated with the results or null if it was not associated with any module.

Parameters
res TestRunResult

Returns
IInvocationContext

getNumAllFailedTestRuns

public int getNumAllFailedTestRuns ()

Return total number of test runs in a failure state

Returns
int

getNumAllFailedTests

public int getNumAllFailedTests ()

Return total number of tests in a failure state (only failed, assumption failures do not count toward it).

Returns
int

getNumTestsInState

public int getNumTestsInState (TestResult.TestStatus status)

Gets the number of tests in given state for this run.

Parameters
status TestResult.TestStatus

Returns
int

getNumTotalTests

public int getNumTotalTests ()

Gets the total number of complete tests for all runs.

Returns
int

getPrimaryBuildInfo

public IBuildInfo getPrimaryBuildInfo ()

Return the primary build info that was reported via invocationStarted(IInvocationContext). Primary build is the build returned by the first build provider of the running configuration. Returns null if there is no context (no build to test case).

Returns
IBuildInfo

getRunResults

public Collection<TestRunResult> getRunResults ()

Gets the results for all test runs.

Returns
Collection<TestRunResult>

getSummary

public TestSummary getSummary ()

Allows the InvocationListener to return a summary.

Returns
TestSummary A TestSummary summarizing the run, or null

hasFailedTests

public boolean hasFailedTests ()

Returns
boolean true if invocation had any failed or assumption failed tests.

hasResultFor

public boolean hasResultFor (String runName)

Returns True if the result map already has an entry for the run name.

Parameters
runName String

Returns
boolean

invocationEnded

public void invocationEnded (long elapsedTime)

Reports that the invocation has terminated, whether successfully or due to some error condition.

Will be automatically called by the TradeFederation framework.

Parameters
elapsedTime long: the elapsed time of the invocation in ms

invocationFailed

public void invocationFailed (Throwable cause)

Reports an incomplete invocation due to some error condition.

Will be automatically called by the TradeFederation framework.

Parameters
cause Throwable: the Throwable cause of the failure

invocationStarted

public void invocationStarted (IInvocationContext context)

Reports the start of the test invocation.

Will be automatically called by the TradeFederation framework. Reporters need to override this method to support multiple devices reporting.

Parameters
context IInvocationContext: information about the invocation

logAssociation

public void logAssociation (String dataName, 
                LogFile logFile)

In some cases, log must be strongly associated with a test cases, but the opportunity to do so on the direct testLogSaved(String, LogDataType, InputStreamSource, LogFile) callback is not possible. Thus, this callback allows to provide a strong association explicitly.

Parameters
dataName String: The name of the data

logFile LogFile: the LogFile that was logged before and should be associated with the test case.

setBuildInfo

public void setBuildInfo (IBuildInfo buildInfo)

This method is deprecated.
Not necessary for testing anymore.

Set the build info. Should only be used for testing.

Parameters
buildInfo IBuildInfo

setLogSaver

public void setLogSaver (ILogSaver logSaver)

Set the ILogSaver to allow the implementor to save files.

Parameters
logSaver ILogSaver: the ILogSaver

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                String trace)

Called when an atomic test flags that it assumes a condition that is false

Parameters
test TestDescription: identifies the test

trace String: stack trace of failure

testEnded

public void testEnded (TestDescription test, 
                HashMap<String, MetricMeasurement.Metric> testMetrics)

Reports the execution end of an individual test case.

If testFailed(TestDescription, String) was not invoked, this test passed. Also returns any key/value metrics which may have been emitted during the test case's execution.

Parameters
test TestDescription: identifies the test

testMetrics HashMap: a Map of the metrics emitted

testEnded

public void testEnded (TestDescription test, 
                long endTime, 
                HashMap<String, MetricMeasurement.Metric> testMetrics)

Alternative to testEnded(TestDescription, Map) where we can specify the end time directly. Combine with testStarted(TestDescription, long) for accurate measure.

Parameters
test TestDescription: identifies the test

endTime long: the time the test ended, measured via System.currentTimeMillis()

testMetrics HashMap: a Map of the metrics emitted

testFailed

public void testFailed (TestDescription test, 
                String trace)

Reports the failure of a individual test case.

Will be called between testStarted and testEnded.

Parameters
test TestDescription: identifies the test

trace String: stack trace of failure

testIgnored

public void testIgnored (TestDescription test)

Called when a test will not be run, generally because a test method is annotated with org.junit.Ignore.

Parameters
test TestDescription: identifies the test

testLog

public void testLog (String dataName, 
                LogDataType dataType, 
                InputStreamSource dataStream)

Provides the associated log or debug data from the test invocation.

Must be called before ITestInvocationListener.invocationFailed(Throwable) or ITestInvocationListener.invocationEnded(long)

The TradeFederation framework will automatically call this method, providing the host log and if applicable, the device logcat.

Parameters
dataName String: a String descriptive name of the data. e.g. "device_logcat". Note dataName may not be unique per invocation. ie implementers must be able to handle multiple calls with same dataName

dataType LogDataType: the LogDataType of the data

dataStream InputStreamSource: the InputStreamSource of the data. Implementers should call createInputStream to start reading the data, and ensure to close the resulting InputStream when complete. Callers should ensure the source of the data remains present and accessible until the testLog method completes.

testLogSaved

public void testLogSaved (String dataName, 
                LogDataType dataType, 
                InputStreamSource dataStream, 
                LogFile logFile)

Called when the test log is saved.

Should be used in place of ITestLogger.testLog(String, LogDataType, InputStreamSource).

Parameters
dataName String: a String descriptive name of the data. e.g. "device_logcat". Note dataName may not be unique per invocation. ie implementers must be able to handle multiple calls with same dataName

dataType LogDataType: the LogDataType of the data

dataStream InputStreamSource: the InputStreamSource of the data. Implementers should call createInputStream to start reading the data, and ensure to close the resulting InputStream when complete.

logFile LogFile: the LogFile containing the meta data of the saved file.

testModuleEnded

public void testModuleEnded ()

Reports the end of a module run.

testModuleStarted

public void testModuleStarted (IInvocationContext moduleContext)

Reports the beginning of a module running. This callback is associated with testModuleEnded() and is optional in the sequence. It is only used during a run that uses modules: ITestSuite based runners.

Parameters
moduleContext IInvocationContext: the IInvocationContext of the module.

testRunEnded

public void testRunEnded (long elapsedTime, 
                HashMap<String, MetricMeasurement.Metric> runMetrics)

Reports end of test run. FIXME: We cannot have two Map<> interfaces with different type, so we have to use HashMap here.

Parameters
elapsedTime long: device reported elapsed time, in milliseconds

runMetrics HashMap: key-value pairs reported at the end of a test run with MetricMeasurement.Metric.

testRunFailed

public void testRunFailed (String errorMessage)

Reports test run failed to complete due to a fatal error.

Parameters
errorMessage String: String describing reason for run failure.

testRunStarted

public void testRunStarted (String name, 
                int numTests)

Reports the start of a test run.

Parameters
name String: the test run name

numTests int: total number of tests in test run

testRunStopped

public void testRunStopped (long elapsedTime)

Reports test run stopped before completion due to a user request.

TODO: currently unused, consider removing

Parameters
elapsedTime long: device reported elapsed time, in milliseconds

testStarted

public void testStarted (TestDescription test, 
                long startTime)

Alternative to testStarted(TestDescription) where we also specify when the test was started, combined with testEnded(TestDescription, long, Map) for accurate measure.

Parameters
test TestDescription: identifies the test

startTime long: the time the test started, measured via System.currentTimeMillis()

testStarted

public void testStarted (TestDescription test)

Reports the start of an individual test case. Older interface, should use testStarted(TestDescription) whenever possible.

Parameters
test TestDescription: identifies the test

Protected methods

setIsAggregrateMetrics

protected void setIsAggregrateMetrics (boolean aggregate)

Toggle the 'aggregate metrics' option

Parameters
aggregate boolean