The VTS Dashboard provides a user backend and user interface for viewing test results from the VTS continuous integration system. It supports test-driven development with tools such as test status notifications to help developers to locate and prevent regression areas during the development cycle (include test monitoring and triaging support).
The user interface supports new features (such as native code coverage) provided by the VTS infrastructure and enables the development of tools with optimized and well-characterized performance by offering continuous performance monitoring.
The following services are required to use the VTS Dashboard:
- Apache Maven, for building and deployment
- Google Cloud App Engine, for web-service hosting
- Google Cloud Datastore, for storage
- Google Stackdriver, for monitoring
Viewing test coverage relies on a REST API to a source code server (e.g. Gerrit), which enables the web service to fetch original source code according to existing access control lists.
The VTS Dashboard uses the following architecture:
Test status results are continuously uploaded to the Cloud Datastore database via a REST interface. The VTS runner automatically processes the results and serializes them using the Protobuf format.
Web servlets form the primary access point for users, delivering and processing data from the Datastore database. The servlets include: a main servlet for delivering all of the tests, a preferences servlet for managing user favorites, a results servlet for populating a test table, a graph servlet for preparing profiling data, and a coverage servlet for preparing coverage data for the client.
Each test module has its own Datastore ancestry tree and test results are indexed with the Unix timestamp of the test start time. Coverage data in the database is stored with the test results as a vector of counts (i.e. for each line in the original source file) and identifying information to fetch the source code from a source code server.
The notification service runs using task queues, identifying test case status changes, and notifying subscribers. Stateful information is stored in a status table to keep track of data freshness and existing failures. This allows for the notification service to provide rich information about individual test case failures and fixes.
VTS Dashboard essential components include the servlets implemented in Java,
the front-end JSPs, CSS stylesheets, and configuration files. The following list
details the locations and descriptions of these components (all paths relative
Settings file where environment variables and dependencies are defined.
Contains endpoints for interacting with the data via REST.
Contains Java models of the Datastore entities.
Contains Java files for Protobuf, including
VtsReportMessage.java, which is a Java implementation of Protobuf type used to describe VTS test results.
Contains Java files for servlets.
Contains Java files for utility functions and classes used by the servlets.
Contains UI tests for the servlets and utils.
Contains files related to the UI (JSP, CSS, XML):
WEB-INF/. Contains configuration and UI files.
jsp/. Contains the JSP files for each web page.
Settings file where environment variables are loaded into variables.
Settings file where servlet mappings and security constraints are defined.
Settings file defining scheduled tasks (i.e. the notifications service).
Setting up the Dashboard
To set up the VTS Dashboard:
- Create a Google Cloud App Engine Project.
- Set up the deployment host by installing:
- Java 8
- Google App Engine SDK
- Generate an OAuth 2.0 Client ID in the Google Cloud API Manager.
- Create a Service Account and create a keyfile.
- Add an email address to the App Engine Email API Authorized Senders List.
- Set up a Google Analytics Account.
- Specify environment variables in the Dashboard
- Set the client ID with the OAuth 2.0 ID (from step 3).
- Set the service client ID with the identifier included in the keyfile (from step 4).
- Specify the sender email address for alerts (from step 5).
- Specify an email domain to which all emails will be sent.
- Specify the address to the Gerrit REST server.
- Specify the OAuth 2.0 scope to use for the Gerrit REST server.
- Specify the Google Analytics ID (from step 6).
- Build and deploy the project.
- In a terminal, run
mvn clean appengine:update
For more information regarding Dashboard setup and configuration, refer to the Android VTS Code Lab.
Robust coverage information requires access to the original source code. However, some code may be sensitive and an additional gateway to it may allow for exploitation of existing access control lists.
This approach does not widen the attack surface because the Dashboard uses the user's cookies to authenticate with an outside service. A user who cannot access source code directly cannot exploit the Dashboard to view sensitive information.