Dexpreopt and <uses-library> checks

Android 12 has build system changes to AOT compilation of DEX files (dexpreopt) for Java modules that have <uses-library> dependencies. In some cases these build system changes can break builds. Use this page to prepare for breakages, and follow the recipes on this page to fix and mitigate them.

Dexpreopt is the process of ahead-of-time compilation of Java libraries and apps. Dexpreopt happens on-host at build time (as opposed to dexopt, which happens on-device). The structure of shared library dependencies used by a Java module (a library or an app) is known as its class loader context (CLC). To guarantee the correctness of dexpreopt, build-time and run-time CLCs must coincide. Build-time CLC is what the dex2oat compiler uses at dexpreopt time (it's recorded in the ODEX files), and run-time CLC is the context in which the precompiled code is loaded on device.

These build-time and run-time CLCs must coincide for reasons of both correctness and performance. For correctness, it's necessary to handle duplicate classes. If the shared library dependencies at runtime are different than those used for compilation, some of the classes might get resolved differently, causing subtle runtime bugs. Performance is also affected by the runtime checks for duplicate classes.

Affected use cases

The first boot is the main use case that's affected by these changes: if ART detects a mismatch between build-time and run-time CLCs, it rejects dexpreopt artifacts and runs dexopt instead. For subsequent boots this is fine because the apps can be dexopted in the background and stored on disk.

Affected areas of Android

This affects all the Java apps and libraries that have runtime dependencies on other Java libraries. Android has thousands of apps, and hundreds of those use shared libraries. Partners are affected as well, as they have their own libraries and apps.

Break changes

The build system needs to know <uses-library> dependencies before it generates dexpreopt build rules. However, it can't access the manifest directly and read the <uses-library> tags in it, because the build system isn't allowed to read arbitrary files when it generates build rules (for performance reasons). Moreover, the manifest might be packaged inside of an APK or a prebuilt. Therefore, the <uses-library> information must be present in the build files (Android.bp or Android.mk).

Previously ART used a workaround that ignored shared library dependencies (known as the &-classpath). This was unsafe and caused subtle bugs, so the workaround was removed in Android 12.

As a result, Java modules that don’t provide correct <uses-library> information in their build files can cause build breakages (caused by a build-time CLC mismatch) or first-boot time regressions (caused by a boot-time CLC mismatch followed by dexopt).

Migration path

Follow these steps to fix a broken build:

  1. Globally disable build-time check for a particular product by setting

    PRODUCT_BROKEN_VERIFY_USES_LIBRARIES := true

    in the product makefile. This fixes build errors (except for special cases, listed in the Fixing breakages section). However, this is a temporary workaround, and it can cause boot-time CLC mismatch followed by dexopt.

  2. Fix the modules that failed before you globally disabled the build-time check by adding the necessary <uses-library> information to their build files (see Fixing breakages for details). For most modules this requires adding a few lines in Android.bp, or in Android.mk.

  3. Disable build-time check and dexpreopt for the problematic cases, on a per-module basis. Disable dexpreopt so you don’t waste build time and storage on artifacts that get rejected at boot.

  4. Globally re-enable the build-time check by unsetting PRODUCT_BROKEN_VERIFY_USES_LIBRARIES that was set in Step 1; the build shouldn't fail after this change (because of steps 2 and 3).

  5. Fix the modules that you disabled in Step 3, one at a time, then re-enable dexpreopt and the <uses-library> check. File bugs if necessary.

Build-time <uses-library> checks are enforced in Android 12.

Fix breakages

The following sections tell you how to fix specific types of breakage.

Build error: CLC mismatch

The build system does a build-time coherence check between the information in Android.bp or Android.mk files and the manifest. The build system can’t read the manifest, but it can generate build rules to read the manifest (extracting it from an APK if necessary), and compare <uses-library> tags in the manifest against the <uses-library> information in the build files. If the check fails, the error looks like this:

error: mismatch in the <uses-library> tags between the build system and the manifest:
    - required libraries in build system: []
                     vs. in the manifest: [org.apache.http.legacy]
    - optional libraries in build system: []
                     vs. in the manifest: [com.x.y.z]
    - tags in the manifest (.../X_intermediates/manifest/AndroidManifest.xml):
        <uses-library android:name="com.x.y.z"/>
        <uses-library android:name="org.apache.http.legacy"/>

note: the following options are available:
    - to temporarily disable the check on command line, rebuild with RELAX_USES_LIBRARY_CHECK=true (this will set compiler filter "verify" and disable AOT-compilation in dexpreopt)
    - to temporarily disable the check for the whole product, set PRODUCT_BROKEN_VERIFY_USES_LIBRARIES := true in the product makefiles
    - to fix the check, make build system properties coherent with the manifest
    - see build/make/Changes.md for details

As the error message suggests, there are multiple solutions, depending on the urgency:

  • For a temporary product-wide fix, set PRODUCT_BROKEN_VERIFY_USES_LIBRARIES := true in the product makefile. The build-time coherence check is still performed, but a check failure doesn't mean a build failure. Instead, a check failure makes the build system downgrade the dex2oat compiler filter to verify in dexpreopt, which disables AOT-compilation entirely for this module.
  • For a quick, global command-line fix, use the environment variable RELAX_USES_LIBRARY_CHECK=true. It has the same effect as does PRODUCT_BROKEN_VERIFY_USES_LIBRARIES, but intended for use on the command-line. The environment variable overrides the product variable.
  • For a solution to root-cause fix the error, make the build system aware of the <uses-library> tags in the manifest. An inspection of the error message shows which libraries cause the problem (as does inspecting AndroidManifest.xml or the manifest inside of an APK that can be checked with `aapt dump badging $APK | grep uses-library`).

For Android.bp modules:

  1. Look for the missing library in the libs property of the module. If it’s there, Soong normally adds such libraries automatically, except in these special cases:

    • The library isn't an SDK library (it's defined as java_library rather than java_sdk_library).
    • The library has a different library name (in the manifest) from its module name (in the build system).

    To fix this temporarily, add provides_uses_lib: "<library-name>" in the Android.bp library definition. For a long-term solution, fix the underlying issue: convert the library to an SDK library, or rename its module.

  2. If the previous step didn't provide a resolution, add uses_libs: ["<library-module-name>"] for required libraries, or optional_uses_libs: ["<library-module-name>"] for optional libraries to the Android.bp definition of the module. These properties accept a list of module names. The relative order of libraries on the list must be the same as the order in the manifest.

For Android.mk modules:

  1. Check if the library has a different library name (in the manifest) from its module name (in the build system). If it does, fix this temporarily by adding LOCAL_PROVIDES_USES_LIBRARY := <library-name> in the Android.mk file of the library, or add provides_uses_lib: "<library-name>" in the Android.bp file of the library (both cases are possible since an Android.mk module might depend on an Android.bp library). For a long-term solution, fix the underlying issue: rename the library module.

  2. Add LOCAL_USES_LIBRARIES := <library-module-name> for required libraries; add LOCAL_OPTIONAL_USES_LIBRARIES := <library-module-name> for optional libraries to the Android.mk definition of the module. These properties accept a list of module names. The relative order of libraries on the list must be the same as in the manifest.

Build error: unknown library path

If the build system can't find a path to a <uses-library> DEX jar (either a build-time path on-host or an install path on-device), it usually fails the build. A failure to find a path can indicate that the library is configured in some unexpected way. Temporarily fix the build by disabling dexpreopt for the problematic module.

Android.bp (module properties):

enforce_uses_libs: false,
dex_preopt: {
    enabled: false,
},

Android.mk (module variables):

LOCAL_ENFORCE_USES_LIBRARIES := false
LOCAL_DEX_PREOPT := false

File a bug to investigate any unsupported scenarios.

Build error: missing library dependency

An attempt to add <uses-library> X from the manifest of module Y to the build file for Y might result in a build error due to the missing dependency, X.

This is a sample error message for Android.bp modules:

"Y" depends on undefined module "X"

This is a sample error message for Android.mk modules:

'.../JAVA_LIBRARIES/com.android.X_intermediates/dexpreopt.config', needed by '.../APPS/Y_intermediates/enforce_uses_libraries.status', missing and no known rule to make it

A common source of such errors is when a library is named differently than its corresponding module is named in the build system. For example, if the manifest <uses-library> entry is com.android.X, but the name of the library module is just X, it causes an error. To resolve this case, tell the build system that the module named X provides a <uses-library> named com.android.X.

This is an example for Android.bp libraries (module property):

provides_uses_lib: “com.android.X”,

This is an example for Android.mk libraries (module variable):

LOCAL_PROVIDES_USES_LIBRARY := com.android.X

Boot-time CLC mismatch

At first boot, search logcat for messages related to CLC mismatch, as shown below:

$ adb wait-for-device && adb logcat \
  | grep -E 'ClassLoaderContext [a-z ]+ mismatch' -A1

The output can have messages of the form shown here:

[...] W system_server: ClassLoaderContext shared library size mismatch Expected=..., found=... (PCL[]... | PCL[]...)
[...] I PackageDexOptimizer: Running dexopt (dexoptNeeded=1) on: ...

If you get a CLC mismatch warning, look for a dexopt command for the faulty module. To fix it, ensure that the build-time check for the module passes. If that doesn't work, then yours might be a special case that isn't supported by the build system (such as an app that loads another APK, not a library). The build system doesn't handle all the cases, because at build time it's impossible to know for certain what the app loads at runtime.

Class loader context

The CLC is a tree-like structure that describes class-loader hierarchy. The build system uses CLC in a narrow sense (it covers only libraries, not APKs or custom-class loaders): it’s a tree of libraries that represents transitive closure of all the <uses-library> dependencies of a library or app. The toplevel elements of a CLC are the direct <uses-library> dependencies specified in the manifest (the classpath). Each node of a CLC tree is a <uses-library> node that might have its own <uses-library> sub-nodes.

Because <uses-library> dependencies are a directed acyclic graph, and not necessarily a tree, CLC can contain multiple subtrees for the same library. In other words, CLC is the dependency graph "unfolded" to a tree. The duplication is only on a logical level; the actual underlying class loaders aren't duplicated (at runtime there’s a single class loader instance for each library).

CLC defines the lookup order of libraries when resolving Java classes used by the library or app. The lookup order is important because libraries can contain duplicate classes, and the class is resolved to the first match.

On device (run-time) CLC

PackageManager (in frameworks/base) creates a CLC to load a Java module on-device. It adds the libraries listed in the <uses-library> tags in the module's manifest as top-level CLC elements.

For each used library, PackageManager gets all its <uses-library> dependencies (specified as tags in the manifest of that library) and adds a nested CLC for each dependency. This process continues recursively until all leaf nodes of the constructed CLC tree are libraries without <uses-library> dependencies.

PackageManager is only aware of shared libraries. The definition of shared in this usage differs from its usual meaning (as in shared vs. static). In Android, Java shared libraries are those listed in XML configs that are installed on-device (/system/etc/permissions/platform.xml). Each entry contains the name of a shared library, a path to its DEX jar file, and a list of dependencies (other shared libraries that this one uses at runtime, and specifies in <uses-library> tags in its manifest).

In other words, there are two sources of information that allow PackageManager to construct CLC at runtime: <uses-library> tags in the manifest, and shared library dependencies in XML configs.

On-host (build-time) CLC

CLC isn't only needed when loading a library or an app, it’s also needed when compiling one. Compilation can happen either on-device (dexopt) or during the build (dexpreopt). Since dexopt takes place on-device, it has the same information as PackageManager (manifests and shared library dependencies). Dexpreopt, however, takes place on-host and in a totally different environment, and it has to get the same information from the build system.

Thus, the build-time CLC used by dexpreopt and the run-time CLC used by PackageManager are the same thing, but computed in two different ways.

Build-time and run-time CLCs must coincide, otherwise the AOT-compiled code created by dexpreopt gets rejected. To check the equality of build-time and run-time CLCs, the dex2oat compiler records build-time CLC in the *.odex files (in the classpath field of the OAT file header). To find the stored CLC, use this command:

oatdump --oat-file=<FILE> | grep '^classpath = '

Build-time and run-time CLC mismatch is reported in logcat during boot. Search for it with this command:

logcat | grep -E 'ClassLoaderContext [a-z ]+ mismatch'

Mismatch is bad for performance, as it forces the library or app to either be dexopted, or to run without optimizations (for example, the app's code might need to be extracted in memory from the APK, a very expensive operation).

A shared library can be either optional or required. From the dexpreopt standpoint, a required library must be present at build time (its absence is a build error). An optional library can be either present or absent at build time: if present, it’s added to the CLC, passed to dex2oat, and recorded in the *.odex file. If an optional library is absent, it’s skipped and isn't added to the CLC. If there’s a mismatch between build-time and run-time status (the optional library is present in one case, but not the other), then the build-time and run-time CLCs don't match and the compiled code gets rejected.

Advanced build system details (manifest fixer)

Sometimes <uses-library> tags are missing from the source manifest of a library or app. This can happen, for example, if one of the transitive dependencies of the library or app starts using another <uses-library> tag, and the library or app's manifest isn't updated to include it.

Soong can compute some of the missing <uses-library> tags for a given library or app automatically, as the SDK libraries in the transitive dependency closure of the library or app. The closure is needed because the library (or app) might depend on a static library that depends on an SDK library, and possibly might again depend transitively through another library.

Not all <uses-library> tags can be computed this way, but when possible, it's prefereable to let Soong add manifest entries automatically; it's less error-prone and simplifies maintenance. For example, when many apps use a static library that adds a new <uses-library> dependency, all the apps must be updated, which is difficult to maintain.