Integrating with Android CTS

Android CTS release packages (available from Android Compatibility Downloads) include deqp tests and require a subset of these tests (known as the mustpass list), to pass. For devices that do not support a target API or extension, tests are skipped and reported as passing.

The mustpass list includes OpenGL ES 3.0, OpenGL ES 3.1, OpenGL ES 3.2, and the Android Extension Pack tests. mustpass files can be found under the android/cts directory in the deqp source tree. You can run deqp tests through the cts-tradefed utility with the following command:

cts-tradefed run cts --plan CTS-DEQP

Duplicating runs without CTS

To replicate the CTS run, install the deqp APK of the CTS package and use the following command:

adb -d shell am start -n com.drawelements.deqp/android.app.NativeActivity -e \
cmdLine "deqp --deqp-case=dEQP-GLES3.some_group.* --deqp-gl-config-name=rgba8888d24s8 --deqp-log-filename=/sdcard/dEQP-Log.qpa

The important part is the --deqp-gl-config-name=rgba8888d24s8 argument, which requests the tests be run on an RGBA 8888 on-screen surface with a 24-bit depth buffer and an 8-bit stencil buffer. Remember to set the desired tests using the --deqp-case argument.

CTS results mapping

In the Android CTS, a test case can end up in one of three states: passed, failed, or not executed (the deqp has more result codes available). CTS automatically maps deqp result codes to CTS results:

  • A CTS pass can include Pass, NotSupported, QualityWarning, and CompatibilityWarning.
  • A CTS failure can include Fail, ResourceError, Crash, Timeout, and InternalError.