Android 14 Camera Image Test Suite Release Notes

The Android 14 release includes a number of changes to Camera ITS.

This page summarizes the Camera ITS changes for Android 14. The changes fall into five broad categories:

• Python and package versions
• Increased LIMITED camera testing
• Increased face-related testing
• Other test changes
• New tests
• Parallel DUT testing

Python and package versions

Android 14 supports the following Python versions in addition to the Python versions and libraries supported in Android 12 and 13:

• Python 3.9.2
• OpenCV 4.2.0
• Numpy 1.20.3
• Matplotlib 3.4.1
• Scipy 1.6.2
• pySerial 3.5
• Pillow 8.3.1
• PyYAML 5.4.1
• Mobly 1.11
• FFmpeg 4.4.1
• (For Display P3 tests only) Colour-science 0.4.2

Increased LIMITED camera testing

In Android 14, the following tests are refactored to enable testing of LIMITED cameras with existing tests.

Scene	Test name	Description
1_1	test_jpeg	Refactored to eliminate the manual_capture() method.
1_2	test_yuv_jpeg_all	Refactored to eliminate the manual_capture() method.
1_2	test_yuv_plus_jpeg	Refactored to eliminate the manual_capture() method.
1_2	test_yuv_plus_raw	Refactored to the manual_capture() method. RAW/RAW10/RAW12 testing merged to a single test.
4	test_multi_camera_alignment	Refactored to the manual_capture() method.

Increased face-related testing

In Android 14, the following face detection tests are refactored.

Scene	Test name	Description
2	test_num_faces	New faces for increased face diversity. Face locations confirmed.

Other test changes

In Android 14, the following tests are refactored to increase test coverage.

Scene	Test name	Description
0	test_read_write	Refactored to flag capture metadata values for sensitivity that fall outside of android.sensor.sensitivityRange.
1_1	test_burst_sameness_manual	Refactored to do two bursts of 50 frames rather than five bursts of 50 frames.
1_2	test_yuv_jpeg_all	Refactored to use the STILL_CAPTURE use case if available.
1_2	test_yuv_plus_jpeg	Refactored to use the STILL_CAPTURE use case if available.
4	test_multi_camera_alignment	Refactored to add YUV captures and to add 640x360 captures to the existing 640x480 captures.

New tests

Android 14 includes the following new tests.

Scene	Test name	Description
2_a	test_autoframing	Verifies that autoframing can adjust the FoV to include all faces with different skin tones regardless of the zoom ratio set.
2_a	test_display_p3	Verifies P3 JPEG output is correct.
2_a	test_preview_min_frame_rate	Verifies that preview FPS reaches minimum defined by camera properties under low light conditions.
3	test_landscape_to_portrait	Verifies landscape to portrait override rotates image properly.
4	test_preview_aspect_ratio_and_crop	Verifies Preview stream aspect ratio, crop, and FoV are appropriate for supported formats.
6	test_in_sensor_zoom	Verifies the camera in-sensor zoom behavior.
6	test_low_latency_zoom	Verifies zoom ratio scales correctly if settings override zoom is set.
6	test_preview_video_zoom_match	Verifies that zoom behaves the same in Preview and Video streams.
extensions	test_hdr_extension	Verifies HDR extension activates correctly when requested.
extensions	test_night_extension	Verifies Night extension activates correctly when requested.
sensor_fusion	test_led_snapshot	Verifies LED snapshots don't saturate or tint image.

Parallel DUT testing

Android 14 supports parallel DUT testing, which lets you test DUTs in parallel with multiple rigs to speed up overall testing. For example, parallel testing lets you test camera 0 in one rig and camera 1 in another rig at the same time.

For more details, see Parallel DUT testing.

Test requirements

Figure 1 shows the Android 14 ITS secret decoder ring. The secret decoder ring shows what test settings individual tests are gated by. Test settings gated by android.request.availableCapabilities are in uppercase letters. The main gating items are:

• MANUAL_SENSOR
• READ_3A
• PER_FRAME_CONTROL
• RAW
• faceDetectModes
• timestampSource
• flash.info.available

Figure 1. Android 14 ITS secret decoder ring