3A Modes and State Transition

While the actual 3A algorithms are up to the HAL implementation, a high-level state machine description is defined by the HAL interface to allow the HAL device and the framework to communicate about the current state of 3A and trigger 3A events.

When the device is opened, all the individual 3A states must be STATE_INACTIVE. Stream configuration does not reset 3A. For example, locked focus must be maintained across the configure() call.

Triggering a 3A action involves simply setting the relevant trigger entry in the settings for the next request to indicate start of trigger. For example, the trigger for starting an autofocus scan is setting the entry ANDROID_CONTROL_AF_TRIGGER to ANDROID_CONTROL_AF_TRIGGER_START for one request; and cancelling an autofocus scan is triggered by setting ANDROID_CONTROL_AF_TRIGGER to ANDROID_CONTRL_AF_TRIGGER_CANCEL. Otherwise, the entry will not exist or be set to ANDROID_CONTROL_AF_TRIGGER_IDLE. Each request with a trigger entry set to a non-IDLE value will be treated as an independent triggering event.

At the top level, 3A is controlled by the ANDROID_CONTROL_MODE setting. It selects between no 3A (ANDROID_CONTROL_MODE_OFF), normal AUTO mode (ANDROID_CONTROL_MODE_AUTO), and using the scene mode setting (ANDROID_CONTROL_USE_SCENE_MODE):

  • In OFF mode, each of the individual auto-focus (AF), auto-exposure (AE), and auto-whitebalance (AWB) modes are effectively OFF, and none of the capture controls may be overridden by the 3A routines.
  • In AUTO mode, AF, AE, and AWB modes all run their own independent algorithms, and have their own mode, state, and trigger metadata entries, as listed in the next section.
  • In USE_SCENE_MODE, the value of the ANDROID_CONTROL_SCENE_MODE entry must be used to determine the behavior of 3A routines. In SCENE_MODEs other than FACE_PRIORITY, the HAL must override the values of ANDROID_CONTROL_AE/AWB/AF_MODE to be the mode it prefers for the selected SCENE_MODE. For example, the HAL may prefer SCENE_MODE_NIGHT to use CONTINUOUS_FOCUS AF mode. Any user selection of AE/AWB/AF_MODE when scene must be ignored for these scene modes.
  • For SCENE_MODE_FACE_PRIORITY, the AE/AWB/AFMODE controls work as in ANDROID_CONTROL_MODE_AUTO, but the 3A routines must bias toward metering and focusing on any detected faces in the scene.

Auto-focus settings and result entries

Main metadata entries
ANDROID_CONTROL_AF_MODE Control for selecting the current autofocus mode. Set by the framework in the request settings.
AF_MODE_OFF AF is disabled; the framework/app directly controls lens position.
AF_MODE_AUTO Single-sweep autofocus. No lens movement unless AF is triggered.
AF_MODE_MACRO Single-sweep up-close autofocus. No lens movement unless AF is triggered
AF_MODE_CONTINUOUS_VIDEO Smooth continuous focusing, for recording video. Triggering immediately locks focus in current position. Canceling resumes continuous focusing.
AF_MODE_CONTINUOUS_PICTURE Fast continuous focusing, for zero-shutter-lag still capture. Triggering locks focus once currently active sweep concludes. Canceling resumes continuous focusing.
AF_MODE_EDOF Advanced extended depth of field focusing. There is no autofocus scan, so triggering one or canceling one has no effect. Images are focused automatically by the HAL.
ANDROID_CONTROL_AF_STATE Dynamic metadata describing the current AF algorithm state, reported by the HAL in the result metadata.
AF_STATE_INACTIVE No focusing has been done, or algorithm was reset. Lens is not moving. Always the state for MODE_OFF or MODE_EDOF. When the device is opened, it must start in this state.
AF_STATE_PASSIVE_SCAN A continuous focus algorithm is currently scanning for good focus. The lens is moving.
AF_STATE_PASSIVE_FOCUSED A continuous focus algorithm believes it is well focused. The lens is not moving. The HAL may spontaneously leave this state.
AF_STATE_PASSIVE_UNFOCUSED A continuous focus algorithm believes it is not well focused. The lens is not moving. The HAL may spontaneously leave this state.
AF_STATE_ACTIVE_SCAN A scan triggered by the user is underway.
AF_STATE_FOCUSED_LOCKED The AF algorithm believes it is focused. The lens is not moving.
AF_STATE_NOT_FOCUSED_LOCKED The AF algorithm has been unable to focus. The lens is not moving.
ANDROID_CONTROL_AFTRIGGER Control for starting an autofocus scan, the meaning of which depends on mode and state. Set by the framework in the request settings.
AF_TRIGGER_IDLE No current trigger.
AF_TRIGGER_START Trigger start of AF scan. Effect depends on mode and state.
AF_TRIGGER_CANCEL Cancel current AF scan if any, and reset algorithm to default.
Additional metadata entries
ANDROID_CONTROL_AF_REGIONS Control for selecting the regions of the field of view (FOV) that should be used to determine good focus. This applies to all AF modes that scan for focus. Set by the framework in the request settings.

Auto-exposure settings and result entries

Main metadata entries
ANDROID_CONTROL_AE_MODE Control for selecting the current auto-exposure mode. Set by the framework in the request settings.
AE_MODE_OFF Autoexposure is disabled; the user controls exposure, gain, frame duration, and flash.
AE_MODE_ON Standard autoexposure, with flash control disabled. User may set flash to fire or to torch mode.
AE_MODE_ON_AUTO_FLASH Standard autoexposure, with flash on at HAL's discretion for precapture and still capture. User control of flash disabled.
AE_MODE_ON_ALWAYS_FLASH Standard autoexposure, with flash always fired for capture, and at HAL's discretion for precapture. User control of flash disabled.
AE_MODE_ON_AUTO_FLASH_REDEYE Standard autoexposure, with flash on at HAL's discretion for precapture and still capture. Use a flash burst at end of precapture sequence to reduce redeye in the final picture. User control of flash disabled.
ANDROID_CONTROL_AE_STATE Dynamic metadata describing the current AE algorithm state, reported by the HAL in the result metadata.
AE_STATE_INACTIVE Initial AE state after mode switch. When the device is opened, it must start in this state.
AE_STATE_SEARCHING AE is not converged to a good value and is adjusting exposure parameters.
AE_STATE_CONVERGED AE has found good exposure values for the current scene, and the exposure parameters are not changing. HAL may spontaneously leave this state to search for a better solution.
AE_STATE_LOCKED AE has been locked with the AE_LOCK control. Exposure values are not changing.
AE_STATE_FLASH_REQUIRED The HAL has converged exposure but believes flash is required for a sufficiently bright picture. Used for determining if a zero-shutter-lag frame can be used.
AE_STATE_PRECAPTURE The HAL is in the middle of a precapture sequence. Depending on AE mode, this mode may involve firing the flash for metering or a burst of flash pulses for redeye reduction.
ANDROID_CONTROL_AE_PRECAPTURE_TRIGGER Control for starting a metering sequence before capturing a high-quality image. Set by the framework in the request settings.
PRECAPTURE_TRIGGER_IDLE No current trigger.
PRECAPTURE_TRIGGER_START Start a precapture sequence. The HAL should use the subsequent requests to measure good exposure/white balance for an upcoming high-resolution capture.
Additional metadata entries
ANDROID_CONTROL_AE_LOCK Control for locking AE controls to their current values.
ANDROID_CONTROL_AE_EXPOSURE_COMPENSATION Control for adjusting AE algorithm target brightness point.
ANDROID_CONTROL_AE_TARGET_FPS_RANGE Control for selecting the target frame rate range for the AE algorithm. The AE routine cannot change the frame rate to be outside these bounds.
ANDROID_CONTROL_AE_REGIONS Control for selecting the regions of the FOV that should be used to determine good exposure levels. This applies to all AE modes besides OFF.

Auto-whitebalance settings and result entries

Main metadata entries
ANDROID_CONTROL_AWB_MODE Control for selecting the current white-balance mode.
AWB_MODE_OFF Auto-whitebalance is disabled. User controls color matrix.
AWB_MODE_AUTO Automatic white balance is enabled; 3A controls color transform, possibly using more complex transforms than a simple matrix.
AWB_MODE_INCANDESCENT Fixed white balance settings good for indoor incandescent (tungsten) lighting, roughly 2700K.
AWB_MODE_FLUORESCENT Fixed white balance settings good for fluorescent lighting, roughly 5000K.
AWB_MODE_WARM_FLUORESCENT Fixed white balance settings good for fluorescent lighting, roughly 3000K.
AWB_MODE_DAYLIGHT Fixed white balance settings good for daylight, roughly 5500K.
AWB_MODE_CLOUDY_DAYLIGHT Fixed white balance settings good for clouded daylight, roughly 6500K.
AWB_MODE_TWILIGHT Fixed white balance settings good for near-sunset/sunrise, roughly 15000K.
AWB_MODE_SHADE Fixed white balance settings good for areas indirectly lit by the sun, roughly 7500K.
ANDROID_CONTROL_AWB_STATE Dynamic metadata describing the current AWB algorithm state, reported by the HAL in the result metadata.
AWB_STATE_INACTIVE Initial AWB state after mode switch. When the device is opened, it must start in this state.
AWB_STATE_SEARCHING AWB is not converged to a good value and is changing color adjustment parameters.
AWB_STATE_CONVERGED AWB has found good color adjustment values for the current scene, and the parameters are not changing. HAL may spontaneously leave this state to search for a better solution.
AWB_STATE_LOCKED AWB has been locked with the AWB_LOCK control. Color adjustment values are not changing.
Additional metadata entries
ANDROID_CONTROL_AWB_LOCK Control for locking AWB color adjustments to their current values.
ANDROID_CONTROL_AWB_REGIONS Control for selecting the regions of the FOV that should be used to determine good color balance. This applies only to auto-whitebalance mode.

General state machine transition notes

Switching between AF, AE, or AWB modes always resets the algorithm's state to INACTIVE. Similarly, switching between CONTROL_MODE or CONTROL_SCENE_MODE if CONTROL_MODE == USE_SCENE_MODE resets all the algorithm states to INACTIVE.

The tables below are per-mode.

AF state machines

mode = AF_MODE_OFF or AF_MODE_EDOF
State Transformation cause New state Notes
INACTIVE AF is disabled
mode = AF_MODE_AUTO or AF_MODE_MACRO
State Transformation cause New state Notes
INACTIVE AF_TRIGGER ACTIVE_SCAN

Start AF sweep

Lens now moving

ACTIVE_SCAN AF sweep done FOCUSED_LOCKED

If AF successful

Lens now locked

ACTIVE_SCAN AF sweep done NOT_FOCUSED_LOCKED

If AF successful

Lens now locked

ACTIVE_SCAN AF_CANCEL INACTIVE

Cancel/reset AF

Lens now locked

FOCUSED_LOCKED AF_CANCEL INACTIVE Cancel/reset AF
FOCUSED_LOCKED AF_TRIGGER ACTIVE_SCAN

Start new sweep

Lens now moving

NOT_FOCUSED_LOCKED AF_CANCEL INACTIVE Cancel/reset AF
NOT_FOCUSED_LOCKED AF_TRIGGER ACTIVE_SCAN

Start new sweep

Lens now moving

All states Mode change INACTIVE
mode = AF_MODE_CONTINUOUS_VIDEO
State Transformation cause New state Notes
INACTIVE HAL initiates new scan PASSIVE_SCAN

Start AF sweep

Lens now moving

INACTIVE AF_TRIGGER NOT_FOCUSED_LOCKED

AF state query

Lens now locked

PASSIVE_SCAN HAL completes current scan PASSIVE_FOCUSED

End AF scan

Lens now locked

PASSIVE_SCAN AF_TRIGGER FOCUSED_LOCKED

Immediate transformation if focus is good

Lens now locked

PASSIVE_SCAN AF_TRIGGER NOT_FOCUSED_LOCKED

Immediate transformation if focus is bad

Lens now locked

PASSIVE_SCAN AF_CANCEL INACTIVE

Reset lens position

Lens now locked

PASSIVE_FOCUSED HAL initiates new scan PASSIVE_SCAN

Start AF scan

Lens now moving

PASSIVE_FOCUSED AF_TRIGGER FOCUSED_LOCKED

Immediate transformation if focus is good

Lens now locked

PASSIVE_FOCUSED AF_TRIGGER NOT_FOCUSED_LOCKED Immediate transformation if focus is bad

Lens now locked

FOCUSED_LOCKED AF_TRIGGER FOCUSED_LOCKED No effect
FOCUSED_LOCKED AF_CANCEL INACTIVE Restart AF scan
NOT_FOCUSED_LOCKED AF_TRIGGER NOT_FOCUSED_LOCKED No effect
NOT_FOCUSED_LOCKED AF_CANCEL INACTIVE Restart AF scan
mode = AF_MODE_CONTINUOUS_PICTURE
State Transformation cause New state Notes
INACTIVE HAL initiates new scan PASSIVE_SCAN

Start AF scan

Lens now moving

INACTIVE AF_TRIGGER NOT_FOCUSED_LOCKED

AF state query

Lens now locked

PASSIVE_SCAN HAL completes current scan PASSIVE_FOCUSED End AF scan

Lens now locked

PASSIVE_SCAN AF_TRIGGER FOCUSED_LOCKED

Eventual transformation once focus good

Lens now locked

PASSIVE_SCAN AF_TRIGGER NOT_FOCUSED_LOCKED

Eventual transformation if cannot focus

Lens now locked

PASSIVE_SCAN AF_CANCEL INACTIVE

Reset lens position

Lens now locked

PASSIVE_FOCUSED HAL initiates new scan PASSIVE_SCAN

Start AF scan

Lens now moving

PASSIVE_FOCUSED AF_TRIGGER FOCUSED_LOCKED

Immediate transformation if focus is good

Lens now locked

PASSIVE_FOCUSED AF_TRIGGER NOT_FOCUSED_LOCKED

Immediate transformation if focus is bad

Lens now locked

FOCUSED_LOCKED AF_TRIGGER FOCUSED_LOCKED No effect
FOCUSED_LOCKED AF_CANCEL INACTIVE Restart AF scan
NOT_FOCUSED_LOCKED AF_TRIGGER NOT_FOCUSED_LOCKED No effect
NOT_FOCUSED_LOCKED AF_CANCEL INACTIVE Restart AF scan

AE and AWB state machines

The AE and AWB state machines are mostly identical. AE has additional FLASH_REQUIRED and PRECAPTURE states. So rows below that refer to those two states should be ignored for the AWB state machine.

mode = AE_MODE_OFF / AWB mode not AUTO
State Transformation cause New state Notes
INACTIVE AE/AWB disabled
mode = AE_MODE_ON_* / AWB_MODE_AUTO
State Transformation cause New state Notes
INACTIVE HAL initiates AE/AWB scan SEARCHING
INACTIVE AE/AWB_LOCK on LOCKED Values locked
SEARCHING HAL finishes AE/AWB scan CONVERGED Good values, not changing
SEARCHING HAL finishes AE scan FLASH_REQUIRED Converged but too dark without flash
SEARCHING AE/AWB_LOCK on LOCKED Values locked
CONVERGED HAL initiates AE/AWB scan SEARCHING Values locked
CONVERGED AE/AWB_LOCK on LOCKED Values locked
FLASH_REQUIRED HAL initiates AE/AWB scan SEARCHING Values locked
FLASH_REQUIRED AE/AWB_LOCK on LOCKED Values locked
LOCKED AE/AWB_LOCK off SEARCHING Values not good after unlock
LOCKED AE/AWB_LOCK off CONVERGED Values good after unlock
LOCKED AE_LOCK off FLASH_REQUIRED Exposure good, but too dark
All AE states PRECAPTURE_START PRECAPTURE Start precapture sequence
PRECAPTURE Sequence done, AE_LOCK off CONVERGED Ready for high-quality capture
PRECAPTURE Sequence done, AE_LOCK on LOCKED Ready for high-quality capture

Enabling manual control

Several controls are also involved in configuring the device 3A blocks to allow for direct application control.

The HAL model for 3A control is that for each request, the HAL inspects the state of the 3A control fields. If any 3A routine is enabled, then that routine overrides the control variables that relate to that routine, and these override values are then available in the result metadata for that capture. So for example, if auto-exposure is enabled in a request, the HAL should overwrite the exposure, gain, and frame duration fields (and potentially the flash fields, depending on AE mode) of the request. The list of relevant controls is:

Control name Unit Notes
android.control.mode enum: OFF, AUTO, USE_SCENE_MODE High-level 3A control. When set to OFF, all 3A control by the HAL is disabled. The application must set the fields for capture parameters itself. When set to AUTO, the individual algorithm controls in android.control.* are in effect, such as android.control.afMode. When set to USE_SCENE_MODE, the individual controls in android.control.* are mostly disabled, and the HAL implements one of the scene mode settings (such as ACTION, SUNSET, or PARTY) as it wishes.
android.control.afMode enum OFF means manual control of lens focusing through android.lens.focusDistance.
android.control.aeMode enum OFF means manual control of exposure/gain/frame duration through android.sensor.exposureTime / .sensitivity / .frameDuration
android.control.awbMode enum OFF means manual control of white balance.