TestRunResult

public class TestRunResult
extends Object

java.lang.Object
   ↳ com.android.tradefed.result.TestRunResult


Holds results from a single test run.

Maintains an accurate count of tests, and tracks incomplete tests.

Not thread safe! The test* callbacks must be called in order

Summary

Fields

public static final String ERROR_DIVIDER

Public constructors

TestRunResult()

Create an emptyTestRunResult.

Public methods

getCompletedTests()

Gets the set of completed tests.

long getElapsedTime()

Returns the current run elapsed time.

int getExpectedTestCount()

Gets the number of test cases this TestRunResult expects to have.

getFailedTests()

Gets the set of failed tests.

String getName()
int getNumAllFailedTests()

Return total number of tests in a failure state (failed, assumption failure)

int getNumCompleteTests()

Gets the number of complete tests in this run ie with status != incomplete.

int getNumTests()

Gets the number of tests in this run.

int getNumTestsInState(TestResult.TestStatus status)

Gets the number of tests in given state for this run.

String getRunFailureMessage()

Return the run failure error message, null if run did not fail.

getRunLoggedFiles()

Returns a copy of the map containing all the logged file associated with that test case.

getRunMetrics()
getRunProtoMetrics()
getTestResults()

Returns a map of the test results.

String getTextSummary()

Returns a user friendly string describing results.

boolean hasFailedTests()
boolean isRunComplete()
boolean isRunFailure()
static TestRunResult merge( testRunResults)
static TestRunResult merge( testRunResults, MergeStrategy strategy)

Merge multiple TestRunResults of the same testRunName.

void resetRunFailure()

Reset the run failure status.

void setAggregateMetrics(boolean metricAggregation)
void setRunComplete(boolean runComplete)
void testAssumptionFailure(TestDescription test, String trace)
void testEnded(TestDescription test, long endTime, testMetrics)
void testEnded(TestDescription test, testMetrics)
void testFailed(TestDescription test, String trace)
void testIgnored(TestDescription test)
void testLogSaved(String dataName, LogFile logFile)

Information about a file being logged are stored and associated to the test case or test run in progress.

void testRunEnded(long elapsedTime, runMetrics)

New interface using the new proto metrics.

void testRunFailed(String errorMessage)
void testRunStarted(String runName, int testCount)

Notify that a test run started.

void testRunStopped(long elapsedTime)
void testStarted(TestDescription test, long startTime)
void testStarted(TestDescription test)

Fields

ERROR_DIVIDER

public static final String ERROR_DIVIDER

Public constructors

TestRunResult

public TestRunResult ()

Create an emptyTestRunResult.

Public methods

getCompletedTests

public  getCompletedTests ()

Gets the set of completed tests.

Returns

getElapsedTime

public long getElapsedTime ()

Returns the current run elapsed time.

Returns
long

getExpectedTestCount

public int getExpectedTestCount ()

Gets the number of test cases this TestRunResult expects to have. The actual number may be less than the expected number due to test crashes. Normally, such a mismatch indicates a test run failure.

Returns
int

getFailedTests

public  getFailedTests ()

Gets the set of failed tests.

Returns

getName

public String getName ()

Returns
String the test run name

getNumAllFailedTests

public int getNumAllFailedTests ()

Return total number of tests in a failure state (failed, assumption failure)

Returns
int

getNumCompleteTests

public int getNumCompleteTests ()

Gets the number of complete tests in this run ie with status != incomplete.

Returns
int

getNumTests

public int getNumTests ()

Gets the number of tests in this run.

Returns
int

getNumTestsInState

public int getNumTestsInState (TestResult.TestStatus status)

Gets the number of tests in given state for this run.

Parameters
status TestResult.TestStatus

Returns
int

getRunFailureMessage

public String getRunFailureMessage ()

Return the run failure error message, null if run did not fail.

Returns
String

getRunLoggedFiles

public  getRunLoggedFiles ()

Returns a copy of the map containing all the logged file associated with that test case.

Returns

getRunMetrics

public  getRunMetrics ()

Returns
a ERROR(/Map) of the test run metrics.

getRunProtoMetrics

public  getRunProtoMetrics ()

Returns
a ERROR(/Map) of the test run metrics with the new proto format.

getTestResults

public  getTestResults ()

Returns a map of the test results.

Returns

getTextSummary

public String getTextSummary ()

Returns a user friendly string describing results.

Returns
String

hasFailedTests

public boolean hasFailedTests ()

Returns
boolean true if test run had any failed or error tests.

isRunComplete

public boolean isRunComplete ()

Returns
boolean true if test run finished.

isRunFailure

public boolean isRunFailure ()

Returns
boolean true if test run failed.

merge

public static TestRunResult merge ( testRunResults)

Parameters
testRunResults

Returns
TestRunResult

merge

public static TestRunResult merge ( testRunResults, 
                MergeStrategy strategy)

Merge multiple TestRunResults of the same testRunName. If a testcase shows up in multiple TestRunResults but has different results (e.g. "boottest-device" runs three times with result FAIL-FAIL-PASS), we concatenate all the stack traces from the FAILED runs and trust the final run result for status, metrics, log files, start/end time.

Parameters
testRunResults : A list of TestRunResult to merge.

strategy MergeStrategy: the merging strategy adopted for merging results.

Returns
TestRunResult the final TestRunResult containing the merged data from the testRunResults.

resetRunFailure

public void resetRunFailure ()

Reset the run failure status.

Resetting the run failure status is sometimes required when retrying. This should be done with care to avoid clearing a real failure.

setAggregateMetrics

public void setAggregateMetrics (boolean metricAggregation)

Parameters
metricAggregation boolean

setRunComplete

public void setRunComplete (boolean runComplete)

Parameters
runComplete boolean

testAssumptionFailure

public void testAssumptionFailure (TestDescription test, 
                String trace)

Parameters
test TestDescription

trace String

testEnded

public void testEnded (TestDescription test, 
                long endTime, 
                 testMetrics)

Parameters
test TestDescription

endTime long

testMetrics

testEnded

public void testEnded (TestDescription test, 
                 testMetrics)

Parameters
test TestDescription

testMetrics

testFailed

public void testFailed (TestDescription test, 
                String trace)

Parameters
test TestDescription

trace String

testIgnored

public void testIgnored (TestDescription test)

Parameters
test TestDescription

testLogSaved

public void testLogSaved (String dataName, 
                LogFile logFile)

Information about a file being logged are stored and associated to the test case or test run in progress.

Parameters
dataName String: the name referencing the data.

logFile LogFile: The LogFile object representing where the object was saved and and information about it.

testRunEnded

public void testRunEnded (long elapsedTime, 
                 runMetrics)

New interface using the new proto metrics.

Parameters
elapsedTime long

runMetrics

testRunFailed

public void testRunFailed (String errorMessage)

Parameters
errorMessage String

testRunStarted

public void testRunStarted (String runName, 
                int testCount)

Notify that a test run started.

Parameters
runName String: the name associated to the test run for tracking purpose.

testCount int: the number of expected test cases associated with the test run.

testRunStopped

public void testRunStopped (long elapsedTime)

Parameters
elapsedTime long

testStarted

public void testStarted (TestDescription test, 
                long startTime)

Parameters
test TestDescription

startTime long

testStarted

public void testStarted (TestDescription test)

Parameters
test TestDescription