This page describes changes added to AOSP to reduce unnecessary file changes between builds. Device implementers who maintain their own build systems can use this information as a guide for reducing the size of their over-the-air (OTA) updates.
Android OTA updates occasionally contain changed files that don't correspond to code changes. They're actually build system artifacts. This can occur when the same code, built at different times, from different directories, or on different machines produces a large number of changed files. Such excess files increase the size of an OTA patch, and make it difficult to determine which code changed.
To make the contents of an OTA more transparent, AOSP includes build system changes designed to reduce the size of OTA patches. Unnecessary file changes between builds have been eliminated, and only patch-related files are contained in OTA updates. AOSP also includes a build diff tool, which filters out common build-related file changes to provide a cleaner build file diff, and a block mapping tool, which helps you keep block allocation consistent.
A build system can create unnecessarily large patches in several ways. To mitigate this, in Android 8.0 and higher, new features were implemented to reduce the patch size for each file diff. Improvements that reduced OTA-update package sizes include the following:
- Usage of Brotli, a generic-purpose, lossless-compression
algorithm for full images on non-A/B device updates. Brotli
can be customized to optimize compression. On larger updates comprised of two
or more blocks in the filesystem (for example,
system.img), device manufacturers or partners can add their own compression algorithms, and can use different compression algorithms on different blocks of the same update.
- Usage of Puffin recompression, a deterministic patching tool for deflate streams, that handles the compression and diff functions for A/B OTA update generation.
- Changes to the delta-generation tool usage, such as how the
bsdifflibrary is used for compressing patches. In Android 9 and higher, the
bsdifftool selects the compression algorithm that would give the best compression results for a patch.
- Improvements to the
update_engineresulted in less memory consumed when patches are applied for A/B device updates.
- Improvements to splitting large zip files for block-based OTA updates. A
imgdiffsplits oversized APK files, based on entry names. This produces a smaller patch compared to splitting files linearly and using the
bsdifftool to compress them.
The following sections discuss various issues that affect OTA-update sizes, their solutions, and examples of implementation in AOSP.
Problem: Filesystems don't guarantee a file order when
asked for a list of files in a directory, though it's commonly the same for the
same checkout. Tools such as
ls sort the results by default, but
the wildcard function used by commands such as
make don't sort. Before using these tools, you must sort the
Solution: When you use tools such as
make with the wildcard function, sort the output of these commands
before using them. When using
Android.mk files, sort them as well. Some tools,
such as Java, do sort inputs, so before you sort the files, verify that the
tool you're using hasn't already done so.
Examples: Many instances were fixed in the core build
system using the builtin
all-*-files-under macro, which includes
all-cpp-files-under (as several definitions were spread out in
other makefiles). For details, refer to the following:
Problem: Changing the directory in which things are built
can cause the binaries to be different. Most paths in the Android build are
relative paths so
__FILE__ in C/C++ isn't a problem. However, the
debug symbols encode the full pathname by default, and the
.note.gnu.build-id is generated from hashing the pre-stripped
binary, so it will change if the debug symbols change.
Solution: AOSP now makes debug paths relative. For details, refer to CL: https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02.
Problem: Timestamps in the build output result in unnecessary file changes. This is likely to happen in the following locations:
__DATE__/__TIME__/__TIMESTAMP__macros in C or C++ code.
- Timestamps embedded in zip-based archives.
__DATE__/__TIME__/__TIMESTAMP__ in C/C++
These macros always produce different outputs for different builds, so don't use them. Here are a few options for eliminating these macros:
- Remove them. For an example, refer to https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f.
- To uniquely identify the running binary, read the build-id from the ELF header.
- To know when the OS was built, read the
ro.build.date(this works for everything except incremental builds, which may not update this date). For an example, refer to https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84.
Embedded timestamps in archives (zip, jar)
Android 7.0 fixed the problem of embedded timestamps in zip archives by
-X to all uses of the
zip command. This
removed the UID/GID of the builder and the extended Unix timestamp from the zip
signapk tool sets timestamps for the APK files that may
vary depending on the server timezone. For details, refer to the CL https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028.
Problem: APK version strings often had the
BUILD_NUMBER appended to their hardcoded versions. Even if nothing
else changed in an APK, as a result, the APK would still be different.
Solution: Remove the build number from the APK version string.
Enable on-device verity computation
If dm-verity is enabled on your device, then OTA tools automatically pick up your verity configuration, and enable on-device verity computation. This allows verity blocks to be computed on android devices, instead of being stored as raw bytes in your OTA package. Verity blocks can use approximately 16MB for a 2GB partition.
However, computing verity on-device can take a long time. Specifically, the Forward Error-correction
code can take a long time. On pixel devices, it tends to take up to 10 minutes. On low-end devices it
could take longer. If you want to disable on-device verity computation, but still enable dm-verity,
you can do so by passing
--disable_fec_computation to the
ota_from_target_files tool when generating an
OTA update. This flag disables on-device verity computation during OTA updates. It decreases OTA installation time,
but increases OTA package size. If your device doesn't have dm-verity enabled, passing this flag
has no effect.
Consistent build tools
Problem: Tools that generate installed files must be consistent (a given input should always produce the same output).
Solutions/Examples: Changes were required in the following build tools:
- NOTICE file creator. The NOTICE file creator was changed to create reproducible NOTICE collections. Refer to CL: https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64.
- Java Android Compiler Kit (Jack). The Jack toolchain required an update to handle occasional changes in generated constructor ordering. Deterministic accessors for constructors was added to the toolchain: https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b.
- ART AOT compiler (dex2oat). The ART compiler binary received an update that added an option to create a deterministic image: https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9.
- The libpac.so file (V8). Every build creates a different
/system/lib/libpac.sofile because the V8 snapshot changes for each build. The solution was to remove the snapshot: https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29.
- Application pre-dexopt (.odex) files. The pre-dexopt (.odex) files contained uninitialized padding on 64-bit systems. This was corrected: https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029.
Use the build diff tool
For cases where it isn't possible to eliminate build-related file changes,
AOSP includes a build diff tool,
target_files_diff.py for use in comparing two file
packages. This tool performs a recursive diff between two builds, excluding
common build-related file changes, such as
- Expected changes in the build output (for example, due to a build number change).
- Changes due to known issues in the current build system.
To use the build diff tool, run the following command:
target_files_diff.py dir1 dir2
dir2 are base directories that contain the
extracted target files for each build.
Keep block allocation consistent
For a given file, although its contents remain the same between two builds, the actual blocks that hold the data might have changed. As a result, the updater must perform unnecessary I/O to move the blocks around for an OTA update.
In a Virtual A/B OTA update, unnecessary I/O can greatly increase the storage space required to store the copy-on-write snapshot. In a non-A/B OTA update, moving the blocks around for an OTA update contributes to update time as there is more I/O due to block moves.
To address this issue, in Android 7.0 Google extended the
make_ext4fs tool for keeping block allocation consistent across
make_ext4fs tool accepts an optional
base_fs flag that attempts to allocate files to the same blocks when
ext4 image. You can extract the block mapping files
(such as the
base_fs map files) from a previous build's target
files' zip file. For each
ext4 partition, there is a
.map file in the
IMAGES directory (for example,
IMAGES/system.map corresponds to the
system partition). These
base_fs files can then be checked-in and
PRODUCT_<partition>_BASE_FS_PATH, as in this example:
PRODUCT_SYSTEM_BASE_FS_PATH := path/to/base_fs_files/base_system.map PRODUCT_SYSTEM_EXT_BASE_FS_PATH := path/to/base_fs_files/base_system_ext.map PRODUCT_VENDOR_BASE_FS_PATH := path/to/base_fs_files/base_vendor.map PRODUCT_PRODUCT_BASE_FS_PATH := path/to/base_fs_files/base_product.map PRODUCT_ODM_BASE_FS_PATH := path/to/base_fs_files/base_odm.map
While this doesn't help reduce the overall OTA package size, it does improve OTA update performance by reducing the amount of I/O. For Virtual A/B updates, it drastically reduces the amount of storage space needed to apply the OTA.
Avoid updating apps
In addition to minimizing build diffs, you can reduce OTA update sizes by excluding updates for apps that get updates through app stores. APKs often comprise a significant part of various partitions on a device. Including the latest versions of apps that get updated by app stores in an OTA update may have a large size impact on OTA packages, and provide little user benefit. By the time users receive an OTA package, they may already have the updated app, or an even newer version, recieved directly from app stores.