Updates made to these display-specific areas are provided below:
Android 10 supports software keyboard for apps running on a non-default display.
Apps running on a non-default display
In terms of which display shows the software keyboard of the Input Method Editor (IME), there are two different modes. The software keyboard is shown on the:
- Same display on which the focused app appears.
- Default display while the focused app is running on a non-default display.
The system determines which mode to use based on the settings of the display on which the focused app appears. For more details, see:
Figure 1. IME software keyboard as it appears on secondary display, including target app
The system uses a single IME, but can shift between displays to follow user focus. Android 10 automatically expects all first- and third-party IMEs to revise the layout and resize according to the new display size when created.
If there's an active connection on display A, and an input field requests input focus on display B, then the following flow occurs:
- A new input connection comes from the input field on display B.
InputMethodManagerServicechecks if the connection should be approved.
- A display is selected for the IME. If display B supports showing the IME and is allowed to show it, then B is used. Otherwise, the primary device display is selected.
- If the selected display is not from display A, then the connection is
InputMethodServiceis destroyed and then created again.
The system won't show an IME on virtual displays that aren't owned by the system. This is due to a security concern that a malicious app could create a virtual display with enabled system decorations support and read user-sensitive information from the surface, such as typing predictions and custom backgrounds.
In Android 9 (and lower), the IME was only available on the default screen, as described in On-Screen Input methods. In Android 10 (and higher), a user can switch between different input text fields on different displays by switching focus, and the IME window moves to the secondary displays.
The implementation in
WindowManager tracks the input method
window (the IME window where the soft keyboard is drawn) and the input method
target (the window where the IME input goes) to manage the IME state.
InputMethodManagerService (IMMS), no other built-in mechanism can
propagate the display change to
InputMethodService (IMS) and
reconfigure the keyboard layout at runtime when moving focus to another display.
To achieve the IME window switch between displays, Android 10 implements the following:
- The IME and input target window are now tracked per display in
DisplayContent#mInputMethodTarget, so that the WindowManager (WM) can manage the IME focus state independently of each display.
- On the IMMS side, when an app client's focus request from the external display
is received through
ViewRootImpl#handleWindowFocusChanged -> InputMethodManager#onPostWindowFocus -> IMMS#startInputOrWindowGainedFocus, it first unbinds the current input method service and then rebinds the service to reattach the new IME window token for the external display in
- On the IMS side, after the
IMS#attachTokenis received, the following flow occurs:
ContextImpl#updateDisplayis called to update the service context's display in
InputMethodService#attachToken(). This calls
ViewGroup#addView()to revise the layout of the keyboard and adapt to the target display checking the current context.
DisplayContent#setInputMethodWindowLocked()is called, the implementation sends process-level display configuration changes using the
WindowProcessControllerto IME process to override resources and display metrics.
InputMethodServiceclient gets the correct configuration with the correct display metrics after
ViewGroup#addView()call to reinitialize the input view.
Multi-session input method editor support
Device implementations with multiple displays expected to be simultaneously used by multiple users to provide appropriate input sources can be configured to simultaneously display multiple input method editors (IME), at most one per display. The following two figures show a sample multi-session IME on two displays:
Figure 2. Sample multi-session IME
Figure 3. Sample multi-session IME
Support for Per-display focus is a prerequisite for this feature. If not, this feature can't be enabled. Due to security restrictions, the per-display focus limitation restricted this feature to a small subset of devices.
In Android 10, support for multi-session IMEs is implemented with separate system services with a different set of APIs and reduced functionality. Multi-session IME isn't compatible with existing IMEs. Either multi-session or single-session service can be used, but not both.
It isn't possible to use existing Android IMEs built on top of the
InputMethodService class because the assumption that a single IME
client can be focused at the same time was made before Android IME APIs were
introduced in Android 1.5 and many public APIs in
InputMethodService have already relied heavily on that assumption.
However, updating the
InputMethodService class to support a
multi-client scenario is challenging because:
- Doing so would introduce an unacceptable amount of complexity into
InputMethodService, which is already hard to maintain.
- IME developers still need to update their implementation to be able to support parallel requests from multiple focused IME clients, which may require a nontrivial redesign on their side (such as input decoder and typing history database).
- Actual use cases for multi-IME clients are expected to evolve rapidly, so the new protocol isn't stable and isn't ready to be exposed as public APIs.
As with single-session (regular) IME, control over showing IME on individual
displays is performed using
There is a sample multi-session IME located at
To test multi-session IME:
- Run these commands:
$ make -j MultiClientInputMethod
$ adb install -r $OUT/system/priv-app/MultiClientInputMethod/MultiClientInputMethod.apk
$ adb root
$ adb shell setprop persist.debug.multi_client_ime \
$ adb reboot
- Try multiple text input scenarios.
MultiClientInputMethodManagerService for implementation