Android Automotive considers voice to be a crucial component in the
enablement of drive-safe interactions and one of the safest ways for users to
interact with the Android Automotive OS while driving. As a result, the existing
Android Voice Assistant APIs (including VoiceInteractionSession
)
have been expanded to enable voice assistants to perform tasks for users
that can be difficult to accomplish while driving.
Tap-to-Read enables voice assistants to read and reply to text messages on
behalf of that user, when the user interacts with message notifications. To provide
this functionality, you can integrate a voice assistant with
CarVoiceInteractionSession
.
In Automotive, notifications posted to the Notification Center identified as INBOX or INBOX_IN_GROUP notifications (for example, SMS messages), will include a Play action button. This button allows the user to have the notification be read aloud by the selected VIA and to optionally reply by voice.
Figure 1. Tap-to-Read notification
Integrate with CarVoiceInteractionSession
1. Support VoiceInteractions
Applications that provide car voice interaction services must
integrate with the existing Android Voice
Interactions (with the exception of VoiceInteractionSession
). While all
other components in the Voice Interaction API remain the same as implemented on mobile
devices, CarVoiceInteractionSession
(described below) replaces
VoiceInteractionSession
. For more information, see these articles:
2. Implement CarVoiceInteractionSession
CarVoiceInteractionSession
exposes APIs that can be used to enable voice assistants to read aloud and then reply
to text messages on behalf of the user.
When comparing CarVoiceInteractionSession
and
VoiceInteractionSession
, the key difference is that
CarVoiceInteractionSession
passes in the action in onShow
so the voice assistant can detect the context of the user's request as soon as
CarVoiceInteractionSession
starts a session.
CarVoiceInteractionSession | VoiceInteractionSession |
---|---|
onShow takes these three parameters:
|
onShow takes these two parameters:
|
Changes in Android 10
Starting with Android 10, the platform calls VoiceInteractionService.onGetSupportedVoiceActions
to detect which actions are supported. The voice assistant overrides and
implements VoiceInteractionService.onGetSupportedVoiceActions
,
as shown below:
public class MyInteractionService extends VoiceInteractionService { private static final ListSUPPORTED_VOICE_ACTIONS = Arrays.asList( CarVoiceInteractionSession.VOICE_ACTION_READ_NOTIFICATION); @Override public Set onGetSupportedVoiceActions(@NonNull Set voiceActions) { Set result = new HashSet<>(voiceActions); result.retainAll(SUPPORTED_VOICE_ACTIONS); return result; } }
Valid actions are described in the table below. For details about each action, see Sequence Diagrams below.
Action | Expected Payload | Expected Voice Interaction action |
---|---|---|
VOICE_ACTION_READ_NOTIFICATION |
Read aloud messages to the user and then fire the Mark as Read pending intent back when the messages are read successfully. Optionally, prompt the user for a reply. | |
VOICE_ACTION_REPLY_NOTIFICATION |
Parcelable with key.KEY_NOTIFICATION
that maps to a StatusBarNotification .Requires android.permission.BIND_NOTIFICATION_LISTENER_SERVICE |
Prompt the user to state the reply message, input the reply message into
the RemoteInputReply of the pending intent, and then fire the
pending intent. |
VOICE_ACTION_HANDLE_EXCEPTION |
String with key. KEY_EXCEPTION that maps to an ExceptionValue (described below). KEY_FALLBACK_ASSISTANT_ENABLED that maps to a Boolean value. If the value
is true, the Fallback Assistant that can handle the user's request has been
disabled. |
The expected action to be taken for the exception will be defined in the documentation for the exception. |
ExceptionValue
EXCEPTION_NOTIFICATION_LISTENER_PERMISSIONS_MISSING
indicates to the voice assistant that it is missing the Manifest.permission.BIND_NOTIFICATION_LISTENER_SERVICE
permission and to get this permission from the user.
3. Request Notification Listener Permission
If the default voice assistant does not have Notification Listener
Permissions, the platform's FallbackAssistant
(if enabled by the car manufacturer) may read the message aloud before the voice assistant is
notifed to request the permission. To determine if the FallbackAssistant is enabled and has,
therefore, read the message, the voice assistant should check the
KEY_FALLBACK_ASSISTANT_ENABLED
Boolean value in the payload.
The platform recommends the voice assistant add rate-limiting logic as to
the number of times this permission is requested. Doing so respects the user who does
not want to grant the voice assistant this permission and prefers the
FallbackAssistant
to read aloud the text messages. Prompting this
user for permission each time the user presses Play on the message notification
could lead to a negative user experience. The platform does not impose rate-limits
on behalf of the voice assistant.
When requesting the Notification Listener permission, the voice assistant should
use CarUxRestrictionsManager
to determine if a user is parked or is driving. If the user is driving, the voice assistant
displays a notification that provides instructions on how to grant the permission. Doing so
helps (and reminds) the user to grant the permission when it is safer.
4. Work with StatusBarNotification
StatusBarNotifications
passed in with the Read and Reply voice
actions are always in a car-compatible messaging notification as described
in Notify
users of messages. While some notifications may not have the Reply Pending
intent, they all have Mark as Read pending intents.
To streamline interactions with notifications, use NotificationPayloadHandler, which provides methods to extract messages from the notification and write the reply messages to the appropriate pending intent of the notification. After the voice assistant reads the message, the voice assistant must fire the Mark as Read intent.
5. Satisfy Tap-to-Read Pre-Conditions
Only the VoiceInteractionSession
of the default voice
assistant is notified when a user triggers the voice action to read and
reply to messages. As mentioned above, this default voice assistant must also
have the Notification Listener permission.
Sequence Diagrams
These figures display the logic flows of CarVoiceInteractionSession actions
.
Figure 2. Sequence diagram for VOICE_ACTION_READ_NOTIFICATION
In the case of Figure 3 below, the application of rate limits on permission requests is recommended.
Figure 3. Sequence diagram for VOICE_ACTION_REPLY_NOTIFICATION
Figure 4. Sequence diagram for VOICE_ACTION_HANDLE_EXCEPTION
Tips and Tricks
Reading the Name of an Application
If your voice assistant would like to read aloud the messaging application's name during the message readout (i.e. "Sam from Hangouts said..."), you should create a function like that shown below to ensure you are reading out the correct name.
@Nullable String getMessageApplicationName(Context context, StatusBarNotification statusBarNotification) { ApplicationInfo info = getApplicationInfo(context, statusBarNotification.getPackageName()); if (info == null) return null; Notification notification = statusBarNotification.getNotification(); // Sometimes system packages will post on behalf of other applications, so check this // field for a system application notification. if (isSystemApp(info) && notification.extras.containsKey(Notification.EXTRA_SUBSTITUTE_APP_NAME)) { return notification.extras.getString(Notification.EXTRA_SUBSTITUTE_APP_NAME); } else { PackageManager pm = context.getPackageManager(); return String.valueOf(pm.getApplicationLabel(info)); } } @Nullable ApplicationInfo getApplicationInfo(Context context, String packageName) { final PackageManager pm = context.getPackageManager(); ApplicationInfo info; try { info = pm.getApplicationInfo(packageName, 0); } catch (PackageManager.NameNotFoundException e) { return null; } return info; } boolean isSystemApp(ApplicationInfo info) { return (info.flags & ApplicationInfo.FLAG_SYSTEM) != 0; }