Workload scheduling

In Android 17 and higher, the Neural Processing Unit (NPU) Manager coordinates model admission control and workload priority across apps and system services. Rather than allowing background jobs to starve latency-sensitive foreground tasks, the npu system service tracks app lifecycle states from ActivityManager and relays effective scheduling rules to the vendor NPU driver through the android.hardware.npu Android Interface Definition Language (AIDL) hardware abstraction layer (HAL).

Framework API for model load arbitration

Before calling low-level neural network libraries or loading large model files, framework clients interact with android.npumanager.NpuManager to reserve model capacity. NpuManager is a system API (@SystemApi), not a public app SDK API, and callers must hold the android.permission.ACCESS_NPU_MODEL_MANAGER_API permission. Clients first build a model load request and then run the asynchronous request and approval flow.

Build a model load request

A model load reservation is represented by ModelLoadRequest. This object contains the following parameters:

  • Unique request identifier
  • Estimated model size class: NPU_MODEL_SIZE_LESS_THAN_1GB, NPU_MODEL_SIZE_BETWEEN_1GB_AND_2GB, or NPU_MODEL_SIZE_GREATER_THAN_2G
  • Intended workload priority, such as NPU_MODEL_PRIORITY_BACKGROUND, NPU_MODEL_PRIORITY_NORMAL, or NPU_MODEL_PRIORITY_OPPORTUNISTIC

The following Java example builds a ModelLoadRequest for a model larger than 2 GB with normal execution priority:

ModelLoadRequest request = new ModelLoadRequest.Builder(requestId)
        .setSize(NPU_MODEL_SIZE_GREATER_THAN_2G)
        .setPriority(NPU_MODEL_PRIORITY_NORMAL)
        .build();

Execute the request and approval flow

Clients submit the reservation asynchronously using the requestCanLoadModel method:

npuManager.requestCanLoadModel(request, callback, executor);

When NPU capacity is evaluated or system pressure changes, the framework notifies the client through ModelLoadRequestCallback with the following events:

  • onCanLoadModel(request, status, listener): Fired when the request is approved. The client receives an NpuManager.ModelLoadStatusListener token. After loading the model into NPU memory, the client must call listener.notifyModelLoaded(request).
  • onRequestUnloadModel(request) or onRequestUnloadModel(request, reason): Fired when the system experiences resource contention, such as an incoming foreground workload or thermal spike, and requires the client to release its model. After releasing NPU resources, the client must call listener.notifyModelUnloaded(request).
  • onModelLoadRequestComplete(request, status): Notifies the client of terminal request lifecycle transitions, such as cancellation.

Clients can cancel pending requests at any time by calling cancelModelLoad(request).

Scheduling HAL integration

Device-specific vendor implementations integrate with the platform scheduler by implementing the IScheduling.aidl service interface in the android.hardware.npu package.

Configure UID priorities and access control

The npu system service pushes per-UID priority values and access rules to the vendor HAL using setSchedulingConfigs and updateSchedulingConfigs with the SchedulingConfig.aidl parcelable:

package android.hardware.npu;

@VintfStability
parcelable SchedulingConfig {
    const int MIN_PRIORITY = 0;
    const int MAX_PRIORITY = 1000;
    int uid;
    int priority;
    boolean hasDirectAccess;
    boolean canAttributeOtherUid;
}

Each field in SchedulingConfig controls how the vendor NPU driver arbitrates work submitted by a user identifier (UID):

Field or constant Description
MIN_PRIORITY and MAX_PRIORITY Valid priority bounds (0 to 1000), where MIN_PRIORITY (0) represents the highest scheduling priority and MAX_PRIORITY (1000) represents the lowest scheduling priority.
uid Linux UID of the client app or system service.
priority Platform importance score in the range 0 to 1000, where 0 is highest priority. Derived from ActivityManager process state so foreground apps outrank cached or background processes.
hasDirectAccess Whether uid is permitted to open direct NPU execution sessions. If false, direct session creation or execution fails with EX_SECURITY.
canAttributeOtherUid Whether a privileged intermediary service, such as AICore, is authorized to attribute NPU work and power usage to a calling client originalUid.

Vendor NPU schedulers must compute an effective priority in the range 0 to 2000 (where 0 is highest priority) by adding SchedulingConfig.priority (0 to 1000) to the client-requested job priority normalized to 0 to 1000, and schedule hardware execution queues according to that effective priority. For example, if a background app submits a high-priority job, its higher priority score lowers its effective priority (producing a higher numeric value) so it doesn't delay foreground camera or UI workloads.

Report task status and attribution

Vendor drivers report the lifecycle of NPU execution groups back to the platform using WorkInfo.aidl:

package android.hardware.npu;

import android.hardware.npu.Uuid;

@VintfStability
parcelable WorkInfo {
    int id;
    @nullable Uuid groupId;
    int uid;
    int debugPid;
    int originalUid;
    @nullable String debugFeatureId;
    int jobPriority;
    int effectivePriority;
    long timestampMs;
    int deviceNumber;
}

When an intermediary service with canAttributeOtherUid enabled executes a model on behalf of a client app, originalUid carries the originating app UID while uid identifies the submitting service. This separation allows platform telemetry and power accounting to bill work to the requesting app instead of the intermediary daemon.

Configure event debouncing

When an ISchedulingCallback listener is registered with IScheduling.setCallback, the HAL debounces related work notifications across at least ISchedulingCallback.DEBOUNCE_DURATION_MS (50 ms). Work items are considered related when they share the same uid, originalUid, and debugFeatureId, or share the same groupId. Debouncing suppresses rapid callback storms during streaming or multi-step inference runs, reducing Binder overhead and log flooding.

Callback lifecycle states are reported through three methods on ISchedulingCallback:

  • onWorkRequested: Workload is received by the vendor service or driver.
  • onWorkStarted: Hardware execution begins, tagged with a StartReason:
    • StartReason.INITIAL: Initial execution run for a new app request.
    • StartReason.RESUMED: Execution resumed after being paused (PAUSED).
  • onWorkEnded: Hardware execution stops after DEBOUNCE_DURATION_MS elapses without related follow-up work, tagged with an EndReason:
    • EndReason.COMPLETED: Execution completed normally.
    • EndReason.CANCELLED_USER: Execution cancelled by the calling app.
    • EndReason.CANCELLED_SYSTEM: Execution preempted and cancelled by higher-priority work.
    • EndReason.PAUSED: Execution temporarily paused to run higher-priority work and eligible to resume with StartReason.RESUMED.
    • EndReason.FAILED: Execution failed due to a driver or hardware error.

Validation

Validate NPU scheduling implementations with the following test suites:

  • Vendor Test Suite (VTS): Run VtsHalNpuSchedulingTargetTest (packaged in VtsHalNpuTargetTest) to verify IScheduling configuration updates, effective priority ordering, UID attribution rules, and ISchedulingCallback debouncing.
  • Compatibility Test Suite (CTS): Run CtsNpuManagerMoblyTestCases to verify end-to-end foreground versus background NPU workload prioritization. For test setup instructions, see Run NPU Manager Mobly tests.