TestRunResult

public class TestRunResult
extends Object

java.lang.Object
   ↳ com.android.tradefed.result.TestRunResult


Holds results from a single test run.

Maintains an accurate count of tests, and tracks incomplete tests.

Not thread safe! The test* callbacks must be called in order

Summary

Fields

public static final String ERROR_DIVIDER

Public constructors

TestRunResult()

Create an emptyTestRunResult.

Public methods

getCompletedTests()

Gets the set of completed tests.

long getElapsedTime()

Returns the current run elapsed time.

int getExpectedTestCount()

Gets the number of test cases this TestRunResult expects to have.

getFailedTests()

Gets the set of failed tests.

String getName()
int getNumAllFailedTests()

Return total number of tests in a failure state (failed, assumption failure)

int getNumCompleteTests()

Gets the number of complete tests in this run ie with status != incomplete.

int getNumTests()

Gets the number of tests in this run.

int getNumTestsInState(TestStatus status)

Gets the number of tests in given state for this run.

int getNumTestsInState(TestResult.TestStatus ddmlibStatus)

FOR COMPATIBILITY with older status.

getPassedTests()

Gets the set of passed tests.

FailureDescription getRunFailureDescription()

Returns the run failure descriptor, null if run did not fail.

String getRunFailureMessage()

Return the run failure error message, null if run did not fail.

MultiMap<String, LogFile> getRunLoggedFiles()

Returns a copy of the map containing all the logged file associated with that test case.

getRunMetrics()
getRunProtoMetrics()
long getStartTime()

Returns the start time of the first testRunStart call.

getTestResults()

Returns a map of the test results.

getTestsInState( statuses)

Gets the set of tests in given statuses.

getTestsResultsInState(TestStatus status)

Returns all the TestResult in a particular state.

String getTextSummary()

Returns a user friendly string describing results.

boolean hasFailedTests()
boolean isRunComplete()
boolean isRunFailure()
static TestRunResult merge( testRunResults)
static TestRunResult merge( testRunResults, MergeStrategy strategy)

Merge multiple TestRunResults of the same testRunName.

void resetRunFailure()

Reset the run failure status.

void setAggregateMetrics(boolean metricAggregation)
void setRunComplete(boolean runComplete)
void testAssumptionFailure(TestDescription test, String trace)
void testAssumptionFailure(TestDescription test, FailureDescription failure)
void testEnded(TestDescription test, long endTime, testMetrics)
void testEnded(TestDescription test, testMetrics)
void testFailed(TestDescription test, FailureDescription failure)
void testFailed(TestDescription test, String trace)
void testIgnored(TestDescription test)
void testLogSaved(String dataName, LogFile logFile)

Information about a file being logged are stored and associated to the test case or test run in progress.

void testRunEnded(long elapsedTime, runMetrics)

New interface using the new proto metrics.

void testRunFailed(FailureDescription failureDescription)
void testRunFailed(String errorMessage)
void testRunStarted(String runName, int testCount, long startTime)

Notify that a test run started.

void testRunStarted(String runName, int testCount)

Notify that a test run started.

void testRunStopped(long elapsedTime)
void testSkipped(TestDescription test, SkipReason reason)
void testStarted(TestDescription test, long startTime)
void testStarted(TestDescription test)

Fields

ERROR_DIVIDER

public static final String ERROR_DIVIDER

Public constructors

TestRunResult

public TestRunResult ()

Create an emptyTestRunResult.

Public methods

getCompletedTests

public  getCompletedTests ()

Gets the set of completed tests.

Returns

getElapsedTime

public long getElapsedTime ()

Returns the current run elapsed time.

Returns
long

getExpectedTestCount

public int getExpectedTestCount ()

Gets the number of test cases this TestRunResult expects to have. The actual number may be less than the expected number due to test crashes. Normally, such a mismatch indicates a test run failure.

Returns
int

getFailedTests

public  getFailedTests ()

Gets the set of failed tests.

Returns

getName

public String getName ()

Returns
String the test run name

getNumAllFailedTests

public int getNumAllFailedTests ()

Return total number of tests in a failure state (failed, assumption failure)

Returns
int

getNumCompleteTests

public int getNumCompleteTests ()

Gets the number of complete tests in this run ie with status != incomplete.

Returns
int

getNumTests

public int getNumTests ()

Gets the number of tests in this run.

Returns
int

getNumTestsInState

public int getNumTestsInState (TestStatus status)

Gets the number of tests in given state for this run.

Parameters
status TestStatus

Returns
int

getNumTestsInState

public int getNumTestsInState (TestResult.TestStatus ddmlibStatus)

FOR COMPATIBILITY with older status. Use getNumTestsInState(com.android.tradefed.result.TestStatus) instead.

Parameters
ddmlibStatus TestResult.TestStatus

Returns
int

getPassedTests

public  getPassedTests ()

Gets the set of passed tests.

Returns

getRunFailureDescription

public FailureDescription getRunFailureDescription ()

Returns the run failure descriptor, null if run did not fail.

Returns
FailureDescription

getRunFailureMessage

public String getRunFailureMessage ()

Return the run failure error message, null if run did not fail.

Returns
String

getRunLoggedFiles

public MultiMap<String, LogFile> getRunLoggedFiles ()

Returns a copy of the map containing all the logged file associated with that test case.

Returns
MultiMap<String, LogFile>

getRunMetrics

public  getRunMetrics ()

Returns
a ERROR(/Map) of the test run metrics.

getRunProtoMetrics

public  getRunProtoMetrics ()

Returns
a ERROR(/Map) of the test run metrics with the new proto format.

getStartTime

public long getStartTime ()

Returns the start time of the first testRunStart call.

Returns
long

getTestResults

public  getTestResults ()

Returns a map of the test results.

Returns

getTestsInState

public  getTestsInState ( statuses)

Gets the set of tests in given statuses.

Parameters
statuses

Returns

getTestsResultsInState

public  getTestsResultsInState (TestStatus status)

Returns all the TestResult in a particular state.

Parameters
status TestStatus

Returns

getTextSummary

public String getTextSummary ()

Returns a user friendly string describing results.

Returns
String

hasFailedTests

public boolean hasFailedTests ()

Returns
boolean true if test run had any failed or error tests.

isRunComplete

public boolean isRunComplete ()

Returns
boolean true if test run finished.

isRunFailure

public boolean isRunFailure ()

Returns
boolean true if test run failed.

merge

public static TestRunResult merge ( testRunResults)

Parameters
testRunResults

Returns
TestRunResult

merge

public static TestRunResult merge ( testRunResults, 
                MergeStrategy strategy)

Merge multiple TestRunResults of the same testRunName. If a testcase shows up in multiple TestRunResults but has different results (e.g. "boottest-device" runs three times with result FAIL-FAIL-PASS), we concatenate all the stack traces from the FAILED runs and trust the final run result for status, metrics, log files, start/end time.

Parameters
testRunResults : A list of TestRunResult to merge.

strategy MergeStrategy: the merging strategy adopted for merging results.

Returns
TestRunResult the final TestRunResult containing the merged data from the testRunResults.

resetRunFailure

public void resetRunFailure ()

Reset the run failure status.

Resetting the run failure status is sometimes required when retrying. This should be done with care to avoid clearing a real failure.

setAggregateMetrics

public void setAggregateMetrics (boolean metricAggregation)

Parameters
metricAggregation boolean

setRunComplete

public void setRunComplete (boolean runComplete)

Parameters
runComplete boolean

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                String trace)

Parameters
test TestDescription

trace String

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                FailureDescription failure)

Parameters
test TestDescription

failure FailureDescription

testEnded

public void testEnded (TestDescription test, 
                long endTime, 
                 testMetrics)

Parameters
test TestDescription

endTime long

testMetrics

testEnded

public void testEnded (TestDescription test, 
                 testMetrics)

Parameters
test TestDescription

testMetrics

testFailed

public void testFailed (TestDescription test, 
                FailureDescription failure)

Parameters
test TestDescription

failure FailureDescription

testFailed

public void testFailed (TestDescription test, 
                String trace)

Parameters
test TestDescription

trace String

testIgnored

public void testIgnored (TestDescription test)

Parameters
test TestDescription

testLogSaved

public void testLogSaved (String dataName, 
                LogFile logFile)

Information about a file being logged are stored and associated to the test case or test run in progress.

Parameters
dataName String: the name referencing the data.

logFile LogFile: The LogFile object representing where the object was saved and and information about it.

testRunEnded

public void testRunEnded (long elapsedTime, 
                 runMetrics)

New interface using the new proto metrics.

Parameters
elapsedTime long

runMetrics

testRunFailed

public void testRunFailed (FailureDescription failureDescription)

Parameters
failureDescription FailureDescription

testRunFailed

public void testRunFailed (String errorMessage)

Parameters
errorMessage String

testRunStarted

public void testRunStarted (String runName, 
                int testCount, 
                long startTime)

Notify that a test run started.

Parameters
runName String: the name associated to the test run for tracking purpose.

testCount int: the number of expected test cases associated with the test run.

startTime long

testRunStarted

public void testRunStarted (String runName, 
                int testCount)

Notify that a test run started.

Parameters
runName String: the name associated to the test run for tracking purpose.

testCount int: the number of expected test cases associated with the test run.

testRunStopped

public void testRunStopped (long elapsedTime)

Parameters
elapsedTime long

testSkipped

public void testSkipped (TestDescription test, 
                SkipReason reason)

Parameters
test TestDescription

reason SkipReason

testStarted

public void testStarted (TestDescription test, 
                long startTime)

Parameters
test TestDescription

startTime long

testStarted

public void testStarted (TestDescription test)

Parameters
test TestDescription