This page describes changes added to AOSP to reduce unnecessary file changes between builds. Device implementers who maintain their own build systems can use this information as a guide for reducing the size of their over-the-air (OTA) updates.
Android OTA updates occasionally contain changed files that don't correspond to code changes. They're actually build system artifacts. This can occur when the same code, built at different times, from different directories, or on different machines produces a large number of changed files. Such excess files increase the size of an OTA patch, and make it difficult to determine which code changed.
To make the contents of an OTA more transparent, AOSP includes build system changes designed to reduce the size of OTA patches. Unnecessary file changes between builds have been eliminated, and only patch-related files are contained in OTA updates. AOSP also includes a build diff tool, which filters out common build-related file changes to provide a cleaner build file diff, and a block mapping tool, which helps you keep block allocation consistent.
A build system can create unnecessarily large patches in several ways. To mitigate this, in Android 8.0 and higher, new features were implemented to reduce the patch size for each file diff. Improvements that reduced OTA-update package sizes include the following:
- Usage of Brotli, a generic-purpose, lossless-compression
algorithm for full images on non-A/B device updates. Brotli
can be customized to optimize compression. On larger updates comprised of two
or more blocks in the filesystem (for example,
system.img
), device manufacturers or partners can add their own compression algorithms, and can use different compression algorithms on different blocks of the same update. - Usage of Puffin recompression, a deterministic patching tool for deflate streams, that handles the compression and diff functions for A/B OTA update generation.
- Changes to the delta-generation tool usage, such as how the
bsdiff
library is used for compressing patches. In Android 9 and higher, thebsdiff
tool selects the compression algorithm that would give the best compression results for a patch. - Improvements to the
update_engine
resulted in less memory consumed when patches are applied for A/B device updates. - Improvements to splitting large zip files for block-based OTA updates. A
mode in
imgdiff
splits oversized APK files, based on entry names. This produces a smaller patch compared to splitting files linearly and using thebsdiff
tool to compress them.
The following sections discuss various issues that affect OTA-update sizes, their solutions, and examples of implementation in AOSP.
File order
Problem: Filesystems don’t guarantee a file order when
asked for a list of files in a directory, though it’s commonly the same for the
same checkout. Tools such as ls
sort the results by default, but
the wildcard function used by commands such as find
and
make
don't sort. Before using these tools, you must sort the
outputs.
Solution: When you use tools such as find
and
make
with the wildcard function, sort the output of these commands
before using them. When using $(wildcard)
or $(shell
find)
in Android.mk
files, sort them as well. Some tools,
such as Java, do sort inputs, so before you sort the files, verify that the
tool you're using hasn't already done so.
Examples: Many instances were fixed in the core build
system using the builtin all-*-files-under
macro, which includes
all-cpp-files-under
(as several definitions were spread out in
other makefiles). For details, refer to the following:
- https://android.googlesource.com/platform/build/+/4d66adfd0e6d599d8502007e4ea9aaf82e95569f
- https://android.googlesource.com/platform/build/+/379f9f9cec4fe1c66b6d60a6c19fecb81b9eb410
- https://android.googlesource.com/platform/build/+/7c3e3f8314eec2c053012dd97d2ae649ebeb5653
- https://android.googlesource.com/platform/build/+/5c64b4e81c1331cab56d8a8c201f26bb263b630c
Build directory
Problem: Changing the directory in which things are built
can cause the binaries to be different. Most paths in the Android build are
relative paths so __FILE__
in C/C++ isn’t a problem. However, the
debug symbols encode the full pathname by default, and the
.note.gnu.build-id
is generated from hashing the pre-stripped
binary, so it will change if the debug symbols change.
Solution: AOSP now makes debug paths relative. For details, refer to CL: https://android.googlesource.com/platform/build/+/6a66a887baadc9eb3d0d60e26f748b8453e27a02.
Timestamps
Problem: Timestamps in the build output result in unnecessary file changes. This is likely to happen in the following locations:
__DATE__/__TIME__/__TIMESTAMP__
macros in C or C++ code.- Timestamps embedded in zip-based archives.
Solutions/Examples: To remove timestamps from the build output, use the instructions given below in __DATE__/__TIME__/__TIMESTAMP__ in C/C++. and Embedded timestamps in archives.
__DATE__/__TIME__/__TIMESTAMP__ in C/C++
These macros always produce different outputs for different builds, so don't use them. Here are a few options for eliminating these macros:
- Remove them. For an example, refer to https://android.googlesource.com/platform/system/core/+/30622bbb209db187f6851e4cf0cdaa147c2fca9f.
- To uniquely identify the running binary, read the build-id from the ELF header.
- To know when the OS was built, read the
ro.build.date
(this works for everything except incremental builds, which may not update this date). For an example, refer to https://android.googlesource.com/platform/external/libchrome/+/8b7977eccc94f6b3a3896cd13b4aeacbfa1e0f84.
Embedded timestamps in archives (zip, jar)
Android 7.0 fixed the problem of embedded timestamps in zip archives by
adding -X
to all uses of the zip
command. This
removed the UID/GID of the builder and the extended Unix timestamp from the zip
file.
A new tool, ziptime
(located in /platform/build/+/master/tools/ziptime/
) resets the
normal timestamps in the zip headers. For details, refer to the README file.
The signapk
tool sets timestamps for the APK files that may
vary depending on the server timezone. For details, refer to the CL https://android.googlesource.com/platform/build/+/6c41036bcf35fe39162b50d27533f0f3bfab3028.
Version strings
Problem: APK version strings often had the
BUILD_NUMBER
appended to their hardcoded versions. Even if nothing
else changed in an APK, as a result, the APK would still be different.
Solution: Remove the build number from the APK version string.
Examples:
- https://android.googlesource.com/platform/packages/apps/Camera2/+/5e0f4cf699a4c7c95e2c38ae3babe6f20c258d27
- https://android.googlesource.com/platform/build/+/d75d893da8f97a5c7781142aaa7a16cf1dbb669c
Enable on-device verity computation
If dm-verity is enabled on your device, then OTA tools automatically pick up your verity configuration, and enable on-device verity computation. This allows verity blocks to be computed on android devices, instead of being stored as raw bytes in your OTA package. Verity blocks can use approximately 16MB for a 2GB partition.
However, computing verity on-device can take a long time. Specifically, the Forward Error-correction
code can take a long time. On pixel devices, it tends to take up to 10 minutes. On low-end devices it
could take longer. If you want to disable on-device verity computation, but still enable dm-verity,
you can do so by passing --disable_fec_computation
to the ota_from_target_files
tool when generating an
OTA update. This flag disables on-device verity computation during OTA updates. It decreases OTA installation time,
but increases OTA package size. If your device doesn't have dm-verity enabled, passing this flag
has no effect.
Consistent build tools
Problem: Tools that generate installed files must be consistent (a given input should always produce the same output).
Solutions/Examples: Changes were required in the following build tools:
- NOTICE file creator. The NOTICE file creator was changed to create reproducible NOTICE collections. Refer to CL: https://android.googlesource.com/platform/build/+/8ae4984c2c8009e7a08e2a76b1762c2837ad4f64.
- Java Android Compiler Kit (Jack). The Jack toolchain required an update to handle occasional changes in generated constructor ordering. Deterministic accessors for constructors was added to the toolchain: https://android.googlesource.com/toolchain/jack/+/056a5425b3ef57935206c19ecb198a89221ca64b.
- ART AOT compiler (dex2oat). The ART compiler binary received an update that added an option to create a deterministic image: https://android.googlesource.com/platform/art/+/ace0dc1dd5480ad458e622085e51583653853fb9.
- The libpac.so file (V8). Every build creates a different
/system/lib/libpac.so
file because the V8 snapshot changes for each build. The solution was to remove the snapshot: https://android.googlesource.com/platform/external/v8/+/e537f38c36600fd0f3026adba6b3f4cbcee1fb29. - Application pre-dexopt’d (.odex) files. The pre-dexopt’d (.odex) files contained uninitialized padding on 64-bit systems. This was corrected: https://android.googlesource.com/platform/art/+/34ed3afc41820c72a3c0ab9770be66b6668aa029.
Using the build diff tool
For cases where it isn't possible to eliminate build-related file changes,
AOSP includes a build diff tool, target_files_diff.py
for use in comparing two file
packages. This tool performs a recursive diff between two builds, excluding
common build-related file changes, such as
- Expected changes in the build output (for example, due to a build number change).
- Changes due to known issues in the current build system.
To use the build diff tool, run the following command:
target_files_diff.py dir1 dir2
dir1
and dir2
are base directories that contain the
extracted target files for each build.
Keeping block allocation consistent
For a given file, although its contents remain the same between two builds, the actual blocks that hold the data might have changed. As a result, the updater must perform unnecessary I/O to move the blocks around for an OTA update.
In a Virtual A/B OTA update, unnecessary I/O can greatly increase the storage space required to store the copy-on-write snapshot. In a non-A/B OTA update, moving the blocks around for an OTA update contributes to update time as there is more I/O due to block moves.
To address this issue, in Android 7.0 Google extended the
make_ext4fs
tool for keeping block allocation consistent across
builds. The make_ext4fs
tool accepts an optional -d
base_fs
flag that attempts to allocate files to the same blocks when
generating an ext4
image. You can extract the block mapping files
(such as the base_fs
map files) from a previous build's target
files' zip file. For each ext4
partition, there is a .map
file in the
IMAGES
directory (for example, IMAGES/system.map
corresponds to the
system
partition). These base_fs
files can then be checked-in and
specified via PRODUCT_<partition>_BASE_FS_PATH
, as in this example:
PRODUCT_SYSTEM_BASE_FS_PATH := path/to/base_fs_files/base_system.map PRODUCT_SYSTEM_EXT_BASE_FS_PATH := path/to/base_fs_files/base_system_ext.map PRODUCT_VENDOR_BASE_FS_PATH := path/to/base_fs_files/base_vendor.map PRODUCT_PRODUCT_BASE_FS_PATH := path/to/base_fs_files/base_product.map PRODUCT_ODM_BASE_FS_PATH := path/to/base_fs_files/base_odm.map
While this doesn’t help reduce the overall OTA package size, it does improve OTA update performance by reducing the amount of I/O. For Virtual A/B updates, it drastically reduces the amount of storage space needed to apply the OTA.
Avoid updating apps
In addition to minimizing build diffs, you can reduce OTA update sizes by excluding updates for apps that get updates through app stores. APKs often comprise a significant part of various partitions on a device. Including the latest versions of apps that get updated by app stores in an OTA update may have a large size impact on OTA packages, and provide little user benefit. By the time users receive an OTA package, they may already have the updated app, or an even newer version, recieved directly from app stores.