Multi-Display Communications API

The Multi-Display Communications API can be used by a system privileged app in AAOS to communicate with the same app (same package name) running in a different occupant zone in a car. This page describes how to integrate the API. To learn more, you can also see CarOccupantZoneManager.OccupantZoneInfo.

Occupant zone

The concept of an occupant zone maps a user to a set of displays. Each occupant zone has a display with the type DISPLAY_TYPE_MAIN. An occupant zone may also have additional displays, such as a cluster display. Each occupant zone is assigned an Android user. Each user has their own accounts and apps.

Hardware configuration

The Comms API supports only a single SoC. In the single SoC model, all occupant zones and users run on the same SoC. The Comms API consists of three components:

• Power management API allows the client to manage the power of the displays in the occupant zones.

• Discovery API allows the client to monitor the states of other occupant zones in the car, and to monitor peer clients in those occupant zones. Use the Discovery API before using the Connection API.

• Connection API allows the client to connect to its peer client in another occupant zone and to send a payload to the peer client.

The Discovery API and the Connection API are required for connection. The Power management API is optional.

The Comms API doesn't support communication between different apps. Instead, it's designed only for communication between apps with the same package name and used only for communication between different visible users.

Integration guide

Implement AbstractReceiverService

To receive the Payload, the receiver app MUST implement the abstract methods defined in AbstractReceiverService. For example:

public class MyReceiverService extends AbstractReceiverService {

    @Override
    public void onConnectionInitiated(@NonNull OccupantZoneInfo senderZone) {
    }

    @Override
    public void onPayloadReceived(@NonNull OccupantZoneInfo senderZone,
            @NonNull Payload payload) {
    }
}

onConnectionInitiated() is invoked when the sender client requests a connection to this receiver client. If user confirmation is needed to establish the connection, MyReceiverService can override this method to launch a permission activity, and call acceptConnection() or rejectConnection() based on the result. Otherwise, MyReceiverService can just call acceptConnection().`

onPayloadReceived()is invoked whenMyReceiverServicehas received aPayloadfrom the sender client.MyReceiverService` can override this method to:

• Forward the Payload to the corresponding receiver endpoint(s), if any. To get the registered receiver endpoints, call getAllReceiverEndpoints(). To forward the Payload to a given receiver endpoint, call forwardPayload()

OR,

• Cache the Payload, and send it when the expected receiver endpoint is registered, for which the MyReceiverService is notified through onReceiverRegistered()

Declare AbstractReceiverService

The receiver app MUST declare the implemented AbstractReceiverService in its manifest file, add an intent filter with action android.car.intent.action.RECEIVER_SERVICE for this service, and require the android.car.occupantconnection.permission.BIND_RECEIVER_SERVICE permission:

<service android:name=".MyReceiverService"
         android:permission="android.car.occupantconnection.permission.BIND_RECEIVER_SERVICE"
         android:exported="true">
    <intent-filter>
        <action android:name="android.car.intent.action.RECEIVER_SERVICE" />
    </intent-filter>
</service>

The android.car.occupantconnection.permission.BIND_RECEIVER_SERVICEpermission ensures that only the framework can bind to this service. If this service doesn't require the permission, a different app might be able to bind to this service and send a Payload to it directly.

Declare permission

The client app MUST declare the permissions in its manifest file.

<!-- This permission is needed for connection API -->
<uses-permission android:name="android.car.permission.MANAGE_OCCUPANT_CONNECTION"/>
<!-- This permission is needed for discovery API -->
<uses-permission android:name="android.car.permission.MANAGE_REMOTE_DEVICE"/>
<!-- This permission is needed if the client app calls CarRemoteDeviceManager#setOccupantZonePower() -->
<uses-permission android:name="android.car.permission.CAR_POWER"/>

Each of the three permissions above are privileged permissions, which MUST be pre-granted by allowlist files. For example, here is the allowlist file of MultiDisplayTest app:

// packages/services/Car/data/etc/com.google.android.car.multidisplaytest.xml
<permissions>
    <privapp-permissions package="com.google.android.car.multidisplaytest">
        … …
        <permission name="android.car.permission.MANAGE_OCCUPANT_CONNECTION"/>
        <permission name="android.car.permission.MANAGE_REMOTE_DEVICE"/>
        <permission name="android.car.permission.CAR_POWER"/>
    </privapp-permissions>
</permissions>

Get Car managers

To use the API, the client app MUST register a CarServiceLifecycleListener to get the associated Car managers:

private CarRemoteDeviceManager mRemoteDeviceManager;
private CarOccupantConnectionManager mOccupantConnectionManager;

private final Car.CarServiceLifecycleListener mCarServiceLifecycleListener = (car, ready) -> {
   if (!ready) {
       Log.w(TAG, "Car service crashed");
       mRemoteDeviceManager = null;
       mOccupantConnectionManager = null;
       return;
   }
   mRemoteDeviceManager = car.getCarManager(CarRemoteDeviceManager.class);
   mOccupantConnectionManager = car.getCarManager(CarOccupantConnectionManager.class);
};

Car.createCar(getContext(), /* handler= */ null, Car.CAR_WAIT_TIMEOUT_WAIT_FOREVER,
       mCarServiceLifecycleListener);

(Sender) Discover

Before connecting to the receiver client, the sender client SHOULD discover the receiver client by registering a CarRemoteDeviceManager.StateCallback:

// The maps are accessed by the main thread only, so there is no multi-thread issue.
private final ArrayMap<OccupantZoneInfo, Integer> mOccupantZoneStateMap = new ArrayMap<>();
private final ArrayMap<OccupantZoneInfo, Integer> mAppStateMap = new ArrayMap<>();

private final StateCallback mStateCallback = new StateCallback() {
        @Override
        public void onOccupantZoneStateChanged(
                @androidx.annotation.NonNull OccupantZoneInfo occupantZone,
                int occupantZoneStates) {
            mOccupantZoneStateMap.put(occupantZone, occupantZoneStates);
        }
        @Override
        public void onAppStateChanged(
                @androidx.annotation.NonNull OccupantZoneInfo occupantZone,
                int appStates) {
            mAppStateMap.put(occupantZone, appStates);
        }
    };

if (mRemoteDeviceManager != null) {
   mRemoteDeviceManager.registerStateCallback(getActivity().getMainExecutor(),
           mStateCallback);
}

Before requesting a connection to the receiver, the sender SHOULD make sure all the flags of the receiver occupant zone and receiver app are set. Otherwise, errors can occur. For example:

private boolean canRequestConnectionToReceiver(OccupantZoneInfo receiverZone) {
    Integer zoneState = mOccupantZoneStateMap.get(receiverZone);
    if ((zoneState == null) || (zoneState.intValue() & (FLAG_OCCUPANT_ZONE_POWER_ON
            // FLAG_OCCUPANT_ZONE_SCREEN_UNLOCKED is not implemented yet. Right now
            // just ignore this flag.
            //  | FLAG_OCCUPANT_ZONE_SCREEN_UNLOCKED
            | FLAG_OCCUPANT_ZONE_CONNECTION_READY)) == 0) {
        return false;
    }
    Integer appState = mAppStateMap.get(receiverZone);
    if ((appState == null) ||
        (appState.intValue() & (FLAG_CLIENT_INSTALLED
            | FLAG_CLIENT_SAME_LONG_VERSION | FLAG_CLIENT_SAME_SIGNATURE
            | FLAG_CLIENT_RUNNING | FLAG_CLIENT_IN_FOREGROUND)) == 0) {
        return false;
    }
    return true;
}

We recommend the sender request a connection to the receiver only when all the flags of the receiver are set. That said, there are exceptions:

• FLAG_OCCUPANT_ZONE_CONNECTION_READY and FLAG_CLIENT_INSTALLED are the minimum requirements needed to establish a connection.

• If the receiver app needs to displays a UI to get user approval of the connection, FLAG_OCCUPANT_ZONE_POWER_ON and FLAG_OCCUPANT_ZONE_SCREEN_UNLOCKED become additional requirements. For a better user experience, FLAG_CLIENT_RUNNING and FLAG_CLIENT_IN_FOREGROUND are also recommended, otherwise the user might be surprised.

• For now (Android 15), FLAG_OCCUPANT_ZONE_SCREEN_UNLOCKED isn't implemented. The client app can just ignore it.

• For now (Android 15), the Comms API only supports multiple users on the same Android instance so that peer apps can have the same long version code (FLAG_CLIENT_SAME_LONG_VERSION) and signature (FLAG_CLIENT_SAME_SIGNATURE). As a result, apps needn't verify that the two values agree.

For a better user experience, the sender client CAN show a UI if a flag is not set. For example, if FLAG_OCCUPANT_ZONE_SCREEN_UNLOCKED is not set, the sender can show a toast or a dialog to prompt the user to unlock the screen of the receiver occupant zone.

When the sender no longer needs to discover the receivers (for example, when it finds all receivers and established connections or becomes inactive), it CAN stop the discovery.

if (mRemoteDeviceManager != null) {
    mRemoteDeviceManager.unregisterStateCallback();
}

When discovery is stopped, existing connections aren't affected. The sender can continue to send Payload to the connected receivers.

(Sender) Request connection

When all flags of the receiver are set, the sender CAN request a connection to the receiver:

    private final ConnectionRequestCallback mRequestCallback = new ConnectionRequestCallback() {
        @Override
        public void onConnected(OccupantZoneInfo receiverZone) {
        }

        @Override
        public void onFailed(OccupantZoneInfo receiverZone, int connectionError) {
        }

        @Override
        public void onDisconnected(OccupantZoneInfo receiverZone) {
        }
    };

if (mOccupantConnectionManager != null && canRequestConnectionToReceiver(receiverZone)) {
    mOccupantConnectionManager.requestConnection(receiverZone,
                getActivity().getMainExecutor(), mRequestCallback);
}

(Receiver service) Accept the connection

Once the sender requests a connection to the receiver, the AbstractReceiverService in the receiver app will be bound by the car service, and AbstractReceiverService.onConnectionInitiated() will be invoked. As explained in the (Sender) Request Connection, onConnectionInitiated() is an abstracted method and MUST be implemented by the client app.

When the receiver accepts the connection request, the sender's ConnectionRequestCallback.onConnected() will be invoked, then the connection is established.

(Sender) Send the payload

Once the connection is established, the sender CAN send Payload to the receiver:

if (mOccupantConnectionManager != null) {
    Payload payload = ...;
    try {
        mOccupantConnectionManager.sendPayload(receiverZone, payload);
    } catch (CarOccupantConnectionManager.PayloadTransferException e) {
        Log.e(TAG, "Failed to send Payload to " + receiverZone);
    }
}

The sender can put a Binder object, or a byte array in the Payload. If the sender needs to send other data types, it MUST serialize the data into a byte array, use the byte array to construct a Payload object, and send the Payload. Then the receiver client gets the byte array from the received Payload, and deserializes the byte array into the expected data object. For example, if the sender wants to send a String hello to the receiver endpoint with ID FragmentB, it can use Proto Buffers to define a data type like this:

message MyData {
  required string receiver_endpoint_id = 1;
  required string data = 2;
}

Figure 1 illustrates the Payload flow:

(Receiver service) Receive and dispatch the payload

Once the receiver app receives the Payload, its AbstractReceiverService.onPayloadReceived() will be invoked. As explained in the Send the payload, onPayloadReceived() is an abstracted method and MUST be implemented by the client app. In this method, the client CAN forward the Payload to the corresponding receiver endpoint(s), or cache the Payload then send it once the expected receiver endpoint is registered.

(Receiver endpoint) Register and unregister

The receiver app SHOULD call registerReceiver() to register the receiver endpoints. A typical use case is that a Fragment needs to receiver Payload, so it registers a receiver endpoint:

private final PayloadCallback mPayloadCallback = (senderZone, payload) -> {
    …
};

if (mOccupantConnectionManager != null) {
    mOccupantConnectionManager.registerReceiver("FragmentB",
                getActivity().getMainExecutor(), mPayloadCallback);
}

Once the AbstractReceiverService in the receiver client dispatches the Payload to the receiver endpoint, the associated PayloadCallback will be invoked.

The client app CAN register multiple receiver endpoints as long as their receiverEndpointIds are unique among the client app. The receiverEndpointId will be used by the AbstractReceiverService to decide which receiver endpoint(s) to dispatch the Payload to. For example:

• The sender specifies receiver_endpoint_id:FragmentB in the Payload. When receiving the Payload, the AbstractReceiverService in the receiver calls forwardPayload("FragmentB", payload) to dispatch the Payload to FragmentB
• The sender specifies data_type:VOLUME_CONTROL in the Payload. When receiving the Payload, the AbstractReceiverService in the receiver knows that this type of Payload should be dispatched to FragmentB, so it calls forwardPayload("FragmentB", payload)

if (mOccupantConnectionManager != null) {
    mOccupantConnectionManager.unregisterReceiver("FragmentB");
}

(Sender) Terminate the connection

Once the sender no longer needs to send Payload to the receiver (for example, it becomes inactive), it SHOULD terminate the connection.

if (mOccupantConnectionManager != null) {
    mOccupantConnectionManager.disconnect(receiverZone);
}

Once disconnected, the sender can no longer send Payload to the receiver.

Connection flow

A connection flow is illustrated in Figure 2.

Troubleshooting

Check the logs

To check the corresponding logs:

1. Run this command for logging:

   adb shell setprop log.tag.CarRemoteDeviceService VERBOSE && adb shell setprop log.tag.CarOccupantConnectionService VERBOSE && adb logcat -s "AbstractReceiverService","CarOccupantConnectionManager","CarRemoteDeviceManager","CarRemoteDeviceService","CarOccupantConnectionService"
   

2. To dump the internal state of CarRemoteDeviceService and CarOccupantConnectionService:

   adb shell dumpsys car_service --services CarRemoteDeviceService && adb shell dumpsys car_service --services CarOccupantConnectionService
   

Null CarRemoteDeviceManager and CarOccupantConnectionManager

Check out these possible root causes:

1. The car service crashed. As illustrated earlier, the two managers are intentionally reset to be null when car service crashes. When car service is restarted, the two managers are set to non-null values.

2. Either CarRemoteDeviceService or CarOccupantConnectionService is not enabled. To determine if one or the other is enabled, run:

   adb shell dumpsys car_service --services CarFeatureController
   

   • Look for mDefaultEnabledFeaturesFromConfig, which should contain car_remote_device_service and car_occupant_connection_service. For example:

     mDefaultEnabledFeaturesFromConfig:[car_evs_service, car_navigation_service, car_occupant_connection_service, car_remote_device_service, car_telemetry_service, cluster_home_service, com.android.car.user.CarUserNoticeService, diagnostic, storage_monitoring, vehicle_map_service]
     

   • By default, these two services are disabled. When a device supports multi-display, you MUST overlay this configuration file. You can enable the two services in a configuration file:

     // packages/services/Car/service/res/values/config.xml
     <string-array translatable="false" name="config_allowed_optional_car_features">
          <item>car_occupant_connection_service</item>
          <item>car_remote_device_service</item>
          … …
     </string-array>
     

Exception when calling the API

If the client app isn't using the API as intended, an exception can occur. In this case, the client app can check the message in the exception and the crash stack to resolve the issue. Examples of misusing the API are:

• registerStateCallback() This client already registered a StateCallback.
• unregisterStateCallback() No StateCallback was registered by this CarRemoteDeviceManager instance.
• registerReceiver() receiverEndpointId is already registered.
• unregisterReceiver() receiverEndpointId is not registered.
• requestConnection() A pending or established connection already exists.
• cancelConnection() No pending connection to cancel.
• sendPayload() No established connection.
• disconnect() No established connection.

Client1 can send Payload to client2, but not the other way around

The connection is one way by design. To establish two-way connection, both client1 and client2 MUST request a connection to each other and then get approval.