2.4 Control Methods

This document provides detailed explanations of the various Screencap (screenshot) and Input (control) methods in MaaFramework and their configurations.

TIP

• For APIs, screencap/input uses int type (bitwise OR combination); for ProjectInterface V2, use string type (use the name directly).
• ProjectInterface V2 supports configuring screencap/input methods for Win32, MacOS, Linux, and other controllers. For Adb controllers, screencap/input uses MaaToolkitAdbDeviceFind to automatically detect and select the optimal method, no manual configuration required.

Adb

Adb Input

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods in a fixed priority order and select the first available method.

By default, all methods except EmulatorExtras are attempted.

Priority: EmulatorExtras > Maatouch > MinitouchAndAdbKey > AdbShell

Name	API Value	Speed	Compatibility	Description
AdbShell	1	Slow	High
MinitouchAndAdbKey	2	Fast	Medium	Key input still uses AdbShell
Maatouch	4	Fast	Medium
EmulatorExtras	8	Fast	Low	Only supports emulators: MuMu 12, Tencent App Store (应用宝)

Adb Screencap

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

By default, all methods except RawByNetcat, MinicapDirect, and MinicapStream are attempted.

MinicapDirect and MinicapStream encode to jpg (lossy compression), which significantly reduces template matching effectiveness and are not recommended.

Name	API Value	Speed	Compatibility	Encoding	Description
EncodeToFileAndPull	1	Slow	High	Lossless
Encode	2	Slow	High	Lossless
RawWithGzip	4	Medium	High	Lossless
RawByNetcat	8	Fast	Low	Lossless
MinicapDirect	16	Fast	Low	Lossy
MinicapStream	32	Very Fast	Low	Lossy
EmulatorExtras	64	Very Fast	Low	Lossless	Only supports emulators: MuMu 12, LDPlayer 9, AVD, and Tencent App Store (应用宝)

Android Native

The Android native controller uses MaaAndroidNativeControlUnit for direct screenshot and input on Android.

Android Native Prerequisites

1. Android platform only

Android Native Configuration

Pass a JSON config to MaaAndroidNativeControllerCreate(config_json):

• library_path: Android native control unit library path
• screen_resolution.width / screen_resolution.height: raw screenshot resolution, also used as the touch coordinate space
• display_id: target display id, optional, defaults to 0
• force_stop: whether to force stop before start_app, optional, defaults to false

NOTE

MaaFramework converts scaled recognition coordinates back to raw screenshot coordinates in ControllerAgent.
The Android native controller no longer performs any frame-to-touch remapping internally, and only clamps to screen_resolution.
If the control unit reports a raw frame resolution different from screen_resolution, screencap fails immediately.

Android Native Input

Multi-touch is supported. contact is the finger id (0 for the first finger, valid range 0–15, matching Android MotionEvent pointer id).
Click / LongPress / Swipe / MultiSwipe / TouchDown / TouchMove / TouchUp can use different contact values to distinguish fingers.

NOTE

The external control unit library's TouchArgs must include a contact field, otherwise multi-touch events cannot be delivered correctly.
pressure is currently not forwarded to the external library.

Win32

Win32 Input

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Different programs on Win32 handle input differently, so there is no universal method.

Name	API Value	Compatibility	Requires Admin	Seizes Cursor	Background	Description
Seize	1	High	No	Yes	No
SendMessage	2	Medium	Maybe	No	Yes
PostMessage	4	Medium	Maybe	No	Yes
LegacyEvent	8	Low	No	Yes	No
PostThreadMessage	16	Low	Maybe	No	Yes	Deprecated
SendMessageWithCursorPos	32	Medium	Maybe	Brief	Yes	Briefly moves cursor to target position then restores, for apps that check real cursor position
PostMessageWithCursorPos	64	Medium	Maybe	Brief	Yes	Briefly moves cursor to target position then restores, for apps that check real cursor position
SendMessageWithWindowPos	128	Medium	Maybe	No	Yes	Briefly moves window to align target with cursor then restores, does not seize mouse
PostMessageWithWindowPos	256	Medium	Maybe	No	Yes	Briefly moves window to align target with cursor then restores, does not seize mouse
Interception	512	Medium	Yes	No	No	Injects mouse and keyboard events through the Interception driver when normal Win32 injection paths do not work
AnchoredTouch	1024	Medium	Maybe	No	Yes	Injects synthetic touch points, the target window receives WM_POINTER messages, mouse only

NOTE

• Admin privileges mainly depend on the target program's permission level. If the target program runs as administrator, running with admin privileges is required for compatibility.
• The WithCursorPos methods briefly move the cursor to the target position, then restore it after sending the message, hence "Brief" cursor seizure, but it does not block user operations.
• The WithWindowPos methods briefly move the window so the target position aligns with the current cursor position, then restore the window position after sending the message. The cursor is not moved, so there is no mouse seizure, but the window may briefly flicker.
• Interception requires the Interception driver to be installed correctly, and the process usually needs the same or higher privilege level as the target application.
• Interception sends mouse and key operations through the driver. Text input still uses system Unicode SendInput and does not go through the Interception driver path.
• AnchoredTouch injects synthetic touch points through InjectSyntheticPointerInput, requires Windows 10 1809 or later. It never moves the cursor and never changes the foreground window, so it does not interfere with the user's work in other windows, and applications that render according to the cursor position are not affected. The cost is a brief visible change on the target window itself while it is occluded, see the next entry.
• AnchoredTouch implements clicking and swiping only. Keyboard operations can be routed to another method, but scroll has no alternative: scrolling always goes through the mouse method, and a synthetic touch device cannot express a wheel. Choose another input method entirely if you need scrolling.
• Synthetic pointers are dispatched by the desktop Z-order, so AnchoredTouch briefly raises the target window to topmost with the lowest layered alpha while the target point is occluded, and restores the original Z-order and alpha once all contacts are released. A partially visible target window flickers for about 70ms per operation. If raising does not take effect, the operation fails instead of injecting into the window that occludes the target.
• Raising requires the WS_EX_LAYERED extended style on the target window. It is added on the first raise, verified before every raise, and removed when the controller goes idle or inactive is called, unless another module is relying on that layered state by then, in which case the style is kept. If the target point is never occluded, no state of the target window is touched at all. If the style cannot be kept or the opacity cannot be lowered, the operation fails rather than raising the target window visibly. Windows that are layered via UpdateLayeredWindow are refused, since touching them would break their painting. WS_EX_LAYERED is documented as incompatible with CS_OWNDC and CS_CLASSDC window classes, but that restriction does not always hold in practice (Unity windows carry CS_OWNDC and work), so such classes only produce a warning and the actual API results decide.
• A minimized target window is not supported, the operation fails with an error: the client area of a minimized window is off-screen and raising does not change that. With the FramePool / PrintWindow screencap methods the window is taken out of the minimized state before every capture, so this does not occur; with the other screencap methods a minimized window cannot be captured either. Note that the pseudo-minimize of those two methods restores the style and opacity on inactive without minimizing the window again.
• While the window is borrowed, WS_EX_TRANSPARENT is temporarily removed, since it lets input pass through to the windows underneath, and restored afterwards. Because the screencap-side pseudo-minimize writes the same window's extended style and opacity, AnchoredTouch verifies before restoring that these attributes still match what it wrote; if they do not, the window is treated as taken over by another module and the old values are not written back, so whatever that module just configured is left intact. This check cannot fully eliminate the window in which the two sides interleave.
• Interception keyboard selection excludes slots without a hardware ID. Automatic selection keeps an attached keyboard bound, or chooses the lowest attached slot when no valid binding remains. This does not identify the keyboard the user last typed on. For multiple keyboards, set MAA_INTERCEPTION_KEYBOARD_DEVICE in the host application's environment before launching it: use a zero-based slot (0–9) or the exact first hardware ID reported in the device initialization log. A hardware ID can follow a keyboard across slot changes; identical devices may share an ID. An unavailable explicit selection fails rather than falling back to another keyboard.
• Keyboard attachment is rechecked before input. Selection changes are deferred until all keys held by the controller have been released. If the bound keyboard disappears while a key is held, release fails rather than sending the release to a replacement device. Stop/reconnect the controller after an input failure. Driver write success alone does not guarantee the target application consumed the event.
• Win32 also provides a Mouse Lock Follow mode: enable with MaaControllerSetOption(ctrl, MaaCtrlOption_MouseLockFollow, &enabled, sizeof(bool)) (set enabled to true to enable, false to disable). Designed for TPS/FPS games that lock the mouse to the window in the background. When enabled, the window continuously follows the mouse cursor, and RawInput counter-moves prevent the game from sensing hardware mouse movement. Use MaaControllerPostRelativeMove to inject intentional camera rotation while this mode is active. Note: On Win32, MaaControllerPostRelativeMove requires mouse-lock-follow mode to be active; calling it without this mode will fail. Only supported with MessageInput-based input methods (SendMessage / PostMessage variants).
• Win32 also provides a Background Managed Keys guardian: declare the managed key domain through MaaControllerSetOption(ctrl, MaaCtrlOption_BackgroundManagedKeys, keycodes, sizeof(int32_t) * count) with an array of virtual key codes to manage. After setting, matching key operations automatically route through the background guardian path and continuously correct key state while the controller is idle.

Win32 Screencap

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

No default value. Client can choose one combination as the default.

Different programs on Win32 handle rendering differently, so there is no universal method.

Name	API Value	Speed	Compatibility	Requires Admin	Background	Description
GDI	1	Fast	Medium	No	No
FramePool	2	Very Fast	Medium	No	Yes	Available on Windows 10 1903+
DXGI_DesktopDup	4	Very Fast	Low	No	No	Desktop duplication (full-screen output copy)
DXGI_DesktopDup_Window	8	Very Fast	Low	No	No	Desktop duplication cropped to window
PrintWindow	16	Medium	Medium	No	Yes
ScreenDC	32	Fast	High	No	No

NOTE

Three predefined combination macros are provided:

• MaaWin32ScreencapMethod_All: all screencap methods
• MaaWin32ScreencapMethod_Foreground: DXGI_DesktopDup_Window | ScreenDC
• MaaWin32ScreencapMethod_Background: FramePool | PrintWindow

FramePool and PrintWindow have built-in pseudo-minimize support: when the target window is minimized, they make it transparent and click-through, then restore it without activation, allowing screencap to continue without disturbing the user.
Other screencap methods will fail when the target window is minimized. Please avoid minimizing the window when using those methods.

MacOS

The MacOS controller is used to control native macOS applications on macOS.

MacOS Prerequisites

1. macOS 14.0 or later
2. The following permissions need to be granted:
   • Screen Recording permission: Required for screenshot functionality
   • Accessibility permission: Required for input control functionality

Permission Debugging

If you encounter permission-related issues, you can reset permissions using the following commands:

# Reset screen recording permission
tccutil reset ScreenCapture

# Reset accessibility permission
tccutil reset Accessibility

TIP

After resetting permissions, you need to restart the application and re-grant the permissions.

TIP

MaaFramework is not responsible for requesting or guiding permissions; clients must implement these functionalities themselves. MaaToolkit provides a set of helper functions for this purpose; please refer to test/macos_test.

MacOS Screencap

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Name	API Value	Speed	Compatibility	Requires Permission	Background	Description
ScreenCaptureKit	1	Fast	High	Screen Recording	Yes	macOS 14.0+

MacOS Input

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Name	API Value	Compatibility	Requires Permission	Background	Description
GlobalEvent	1	High	Accessibility	No	Injects into the global HID event stream via CGEventPost(kCGHIDEventTap), dispatched by the OS to the front window (automatically activates the target window)
PostToPid	2	Medium	Accessibility	Yes	Sends directly to the target process via CGEventPostToPid, no need for the target window to be in the foreground

NOTE

• Screencap: ScreenCaptureKit supports background capture, including fullscreen windows on other Spaces, without requiring the target window to be in the foreground. Does not support Stage Manager, screenshots only capture the tilted thumbnail window, suspected system bug.
• Keyboard input: In PostToPid mode, keyboard input supports background operation without activating the target window.
• Mouse input: PostToPid supports background and fullscreen input and clicking. However, some games (e.g., 异环) check for window focus on the first click and steal focus if absent, causing the window to repeatedly jump to the foreground and making it impossible to stop. It is recommended that the frontend prompt users to run in windowed mode or provide a hotkey to stop execution.

PlayCover (macOS)

The PlayCover controller is used to control iOS applications running via fork PlayCover on macOS.

PlayCover Prerequisites

1. Install fork PlayCover on macOS
2. Target iOS app must have MaaTools feature enabled in PlayCover

Gamepad (Windows)

The Gamepad controller is used to emulate Xbox 360 or DualShock 4 gamepad input on Windows, suitable for games that require gamepad control.

Gamepad Prerequisites

ViGEm Bus Driver must be installed.

Gamepad Types

Type	API Value	Description
Xbox360	0	Microsoft Xbox 360 Controller (wired)
DualShock4	1	Sony DualShock 4 Controller (wired)

Control Mapping

The Gamepad controller uses the following mapping:

Digital Buttons (click_key/key_down/key_up)

Use MaaGamepadButton_* constants as key values. Xbox button values are used for Xbox360 gamepad, DS4 face buttons automatically map to Xbox equivalents:

Button	API Value	Xbox 360	DualShock 4 Equivalent
DPAD_UP	1	D-pad Up	D-pad Up
DPAD_DOWN	2	D-pad Down	D-pad Down
DPAD_LEFT	4	D-pad Left	D-pad Left
DPAD_RIGHT	8	D-pad Right	D-pad Right
START / OPTIONS	16	Start	Options
BACK / SHARE	32	Back	Share
LEFT_THUMB / L3	64	Left Stick Press	L3
RIGHT_THUMB / R3	128	Right Stick Press	R3
LB / L1	256	LB	L1
RB / R1	512	RB	R1
GUIDE	1024	Xbox Guide	-
A / CROSS	4096	A	✗ (Cross)
B / CIRCLE	8192	B	○ (Circle)
X / SQUARE	16384	X	□ (Square)
Y / TRIANGLE	32768	Y	△ (Triangle)
PS	65536	-	PS (DS4 only)
TOUCHPAD	131072	-	Touchpad Press (DS4 only)

Analog Input (touch_down/touch_move/touch_up)

Use the contact parameter to select the control target:

Contact	Description	x/y Range	pressure Range
0	Left Stick	-32768 ~ 32767	Ignored
1	Right Stick	-32768 ~ 32767	Ignored
2	Left Trigger (LT/L2)	Ignored	0 ~ 255
3	Right Trigger (RT/R2)	Ignored	0 ~ 255

NOTE

• ViGEm Bus Driver must be installed to use this controller.

Linux

Linux Screencap

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Method	API Value	Requires Permission	Description
Wlr	1	No	Screencap using wlr-screencopy-unstable-v1 protocol
PipeWire	4	No	Screencap using PipeWire

TIP

MaaFramework is not responsible for handling the Screencast Portal; clients must implement these functionalities themselves. MaaToolkit provides a set of helper functions for this purpose; please refer to test/linux_test.

Linux Input

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Method	API Value	Requires Permission	Description
Wlr	1	No	Input using virtual-keyboard-unstable-v1 and wlr-virtual-pointer-unstable-v1 protocol
UInput	2	Yes	Input using /dev/uinput
Libei	4	No	Input using libei (EIS socket), e.g. the gamescope-<n>-ei socket provided by gamescope

Environment Prerequisites

When using Wlr screencap method, the wlroots compositor must support the following protocols:

• Wayland core protocol
• wlr-screencopy-unstable-v1

When using Wlr input method, the wlroots compositor must support the following protocols:

• Wayland core protocol
• virtual-keyboard-unstable-v1
• wlr-virtual-pointer-unstable-v1

When using PipeWire screencap method, there are two modes:

• Monitor capture: pass the compositor's PipeWire FD (pw_socket_fd) and node ID (pw_node_id) obtained from xdg-desktop-portal.
• Session-daemon node capture: no FD; set pw_node_id to attach to a node on the session PipeWire daemon (e.g. gamescope's Video/Source node). Discover the node ID with MaaToolkitGamescopeInstanceFindAll.

In ProjectInterface V2, select the mode via controller.linux.pipewire_source: Portal corresponds to monitor capture, Gamescope to session-daemon node capture (default Gamescope).

When using Libei input method, eis_socket_path must point to the EIS socket provided by the compositor, e.g. gamescope's /run/user/<uid>/gamescope-<n>-ei. Text input requires libei >= 1.6.0 on the system; older distributions such as Ubuntu 24.04 only ship 1.2.1 and need a manual upgrade.

When using UInput input method, the current user must have read/write permission for the /dev/uinput device.

When using UInput input method, it is recommended to configure an udev rule: create /etc/udev/rules.d/99-uinput.rules with the following content: KERNEL=="uinput", MODE="0660", GROUP="input" and add the current user to the input group, then reboot for the changes to take effect.

NOTE

It is recommended to start a nested compositor session when using Wlr input method. It is not recommended to control the compositor managing the current desktop, as this may cause unexpected behavior during task execution.

Keyboard Input

Under the Linux controller, keycodes passed to MaaControllerPostKey{Down,Up} are by default evdev scan codes, defined in linux/input-event-codes.h.

If you prefer using Win32 Virtual-Key codes (VK_*), set the use_win32_vk_code field in the config_json parameter of MaaLinuxControllerCreate to true. When enabled, keycodes passed to MaaControllerPostClickKey / MaaControllerPostKey{Down,Up} are interpreted as Win32 VK codes and translated to evdev codes internally. This flag is decided at creation time and cannot be changed at runtime; it makes it easy to reuse key handling logic written for the Win32 controller.

NOTE

• Text Input: MaaControllerPostInputText only supports ASCII characters.