Device manufacturers can implement the CameraX OEM vendor library to expose device-specific effects to third-party developers through the CameraX extensions interface. CameraX defines the CameraX extensions interface for vendor-implemented classes loaded at runtime. This page describes how to implement the OEM vendor library and enable it on devices.
Before implementing the vendor library, make sure you understand how the CameraX Jetpack support library works. To learn more about CameraX, see CameraX overview. For more information on vendor extensions, see Vendor extensions.
Architecture
Figure 1. Vendor extensions architecture diagram
This diagram describes the architecture of the CameraX vendor extensions. Third-party apps can be built against the CameraX extensions library (camera-extensions) and use the camera extensions public API (camera-extensions API). The camera-extensions API is defined by CameraX and is updated when a new version of the camera-extensions library is released. The versioning of the camera-extensions public API and camera-extensions library are the same.
From Android 12, third-party apps can be built against
the Camera2 extensions API. To enable the supported extension modes, apps can
create a camera capture session
(CameraExtensionSession
)
using
CameraDevice#createExtensionSession()
.
The extensions interface (extensions-interface) defined by CameraX allows both the CameraX camera-extensions library and the Camera2 extensions API to communicate with the OEM vendor library (camera-extensions-stub). This means that devices that implement the CameraX OEM vendor library also have support for the Camera2 extensions API.
Implementing the OEM vendor library
These implementation instructions use the bokeh (portrait) vendor extension as
an example, but you can apply these to other extensions such as the HDR, face
retouch, and night mode extensions. To do this, copy and paste the code used for
the bokeh extension and replace the extension name with the desired extension
(for example, replacing BokehImageCaptureExtenderImpl
with
HdrImageCaptureExtenderImpl
).
You aren't required to provide an implementation for every effect. An effect without a vendor implementation defaults to the CameraX implementation. If an effect isn't available, the default implementation doesn't enable the effect and reports to the third-party developer that the effect is unavailable.
Version verification
When loading the OEM library, CameraX verifies that the OEM library contains a version that's compatible with the extensions interface version (referred to as extension-version in this document). To determine version compatibility, CameraX checks only the major and minor versions (for example, 1.0) but doesn't check the patch version because that's used only for bug fixes, not interface changes. The required extension-version for specific APIs is noted in this document. APIs without a specified extension-version are compatible with extension-version 1.0.
To verify the version, CameraX queries the
ExtensionVersionImpl
interface. CameraX then uses the version reported by the OEM library to
determine the functionality that can be called.
Initialization
CameraX starts the initialization process when it has determined the version of
the extensions interface implemented by the OEM library. The
InitializerImpl.init
method signals to the OEM library that an app wants to
use extensions. No other calls to the OEM library (aside from version checking)
are made until OnExtensionsInitializedCallback
is called with a success
state.
This method must be implemented as of extension-version 1.1. For details, see
the source at
InitializerImpl
.
ExtenderStateListener interface
CameraX provides hooks into several places in its pipeline to allow the OEM
library to set relevant SessionParameters
and CaptureRequest
values. To
allow the OEM library to set these values at the specified times, implement the
ExtenderStateListener
interface. This interface must be implemented as part of any extender, whether
it's a preview, image capture, bokeh, or HDR extender.
Image capture
To support the extension for
image capture,
implement the corresponding ImageCaptureExtender
interface (for example,
BokehImageCaptureExtenderImpl
or
HdrImageCaptureExtenderImpl
).
ImageCaptureExtenderImpl
includes interfaces required for extensions related to image capture.
The
CaptureProcessorImpl
interface must be implemented for post processing to be done at the app layer.
As of extensions interface version 1.1, only the YUV_420_888
input image
format must be supported. The CaptureProcessor
interface isn't required if
processing is done in the camera HAL.
The following diagram illustrates the image capture process flow.
Figure 2. Image capture flow diagram
Example: BokehImageCaptureExtenderImpl
To support the bokeh extension for image capture, implement the
BokehImageCaptureExtenderImpl
class in the androidx.camera.extensions.impl
package.
Preview
To support the extension for
preview,
implement the corresponding PreviewExtender
interface (for example,
BokehPreviewExtenderImpl
or
HdrPreviewExtenderImpl
).
The PreviewExtender
interface includes the interfaces required for
preview-related extensions.
For details, see the source at
PreviewExtenderImpl
.
Image processing for the preview extension can be performed in the camera HAL or
the app layer. This is determined by the value of ProcessorType
, which is
returned by PreviewExtenderImpl
.
If the PROCESSOR_TYPE_REQUEST_UPDATE_ONLY
type is returned, processing is
performed in the HAL through CaptureRequest
keys. If the
PROCESSOR_TYPE_IMAGE_PROCESSOR
type is returned, processing is performed in
the app layer by the
PreviewImageProcessorImpl
interface. This interface operates on an Image
and TotalCaptureResult
pair.
As of extensions interface version 1.1, only the YUV_420_888
input image
format must be supported.
The following diagram illustrates the process flow for the preview vendor extension.
Figure 3. Preview flow diagram
Example: BokehPreviewExtenderImpl
To support bokeh for preview, implement the
BokehPreviewExtenderImpl
class in the androidx.camera.extensions.impl
package.
Reference implementation
For a reference OEM vendor library implementation, see
camera-testlib-extensions
.
Note that this implementation performs passthroughs without actually
implementing the effects.
Setting up the vendor library on a device
The OEM vendor library isn't built into an app but instead is loaded from the
device at runtime by CameraX. The <uses-library>
tag declares that the
androidx.camera.extensions.impl
library, which is defined in the
AndroidManifest.xml
file, is a dependency of CameraX and must be loaded at runtime. This allows
third-party apps using vendor extensions to automatically attempt to load the
OEM vendor library. The OEM library is marked as optional so apps can run on
devices that don't have the library on the device.
CameraX handles this behavior automatically when an app tries to use a vendor extension as long as the device manufacturer places the OEM library on the device so that it can be discovered by the app.
To set up the OEM library on a device, do the following:
- Add a permission file, which is required by the
<uses-library>
tag, using the following format:/etc/permissions/ANY_FILENAME.xml
. For example,/etc/permissions/camera_extensions.xml
. The files in this directory provide a mapping of the library named in<uses-library>
to the actual file path on the device. Use the example below to add the required information to the file.
name
must beandroidx.camera.extensions.impl
as that's the library that CameraX searches for.file
is the absolute path of the file that contains the extensions implementation (for example,/system/framework/androidx.camera.extensions.impl.jar
).
<?xml version="1.0" encoding="utf-8"?> <permissions> <library name="androidx.camera.extensions.impl" file="OEM_IMPLEMENTED_JAR" /> </permissions>
In Android 12 or higher, devices supporting CameraX
extensions must have the ro.camerax.extensions.enabled
property set to true
,
which allows for querying whether a device supports extensions.
To do this, add the following line in the device make file:
PRODUCT_VENDOR_PROPERTIES += \
ro.camerax.extensions.enabled=true \
Validation
To test your implementation of the OEM vendor library during the
development stage, use the example app at
androidx-main/camera/integration-tests/extensionstestapp/
,
which runs through various vendor extensions.
After you complete your implementation, use the CameraX Vendor Extensions Validation Tool to run automated and manual tests to verify that the vendor library is implemented correctly.
Frequently asked questions (FAQ)
Are there any restrictions on API levels?
Yes. This depends on the Android API feature set that's required by the OEM
vendor library implementation. For example,
ExtenderStateListener.onPresetSession()
uses the
SessionConfiguration.setSessionParameters()
call to set a baseline set of tags. This call is available only on API level
28 and higher. For details on specific interface methods, see the
API reference documentation.