2.4 Control Methods

This document provides detailed explanations of the various Screencap (screenshot) and Input (control) methods in MaaFramework and their configurations.

TIP

• For APIs, screencap/input uses int type (bitwise OR combination); for ProjectInterface V2, use string type (use the name directly).
• ProjectInterface V2 only supports configuring screencap/mouse/keyboard methods for Win32 controllers. For Adb controllers, screencap/input uses MaaToolkitAdbDeviceFind to automatically detect and select the optimal method, no manual configuration required.

Adb

Adb Input

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods in a fixed priority order and select the first available method.

By default, all methods except EmulatorExtras are attempted.

Priority: EmulatorExtras > Maatouch > MinitouchAndAdbKey > AdbShell

Name	API Value	Speed	Compatibility	Description
AdbShell	1	Slow	High
MinitouchAndAdbKey	2	Fast	Medium	Key input still uses AdbShell
Maatouch	4	Fast	Medium
EmulatorExtras	8	Fast	Low	Only supports emulators: MuMu 12

Adb Screencap

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

By default, all methods except RawByNetcat, MinicapDirect, and MinicapStream are attempted.

MinicapDirect and MinicapStream encode to jpg (lossy compression), which significantly reduces template matching effectiveness and are not recommended.

Name	API Value	Speed	Compatibility	Encoding	Description
EncodeToFileAndPull	1	Slow	High	Lossless
Encode	2	Slow	High	Lossless
RawWithGzip	4	Medium	High	Lossless
RawByNetcat	8	Fast	Low	Lossless
MinicapDirect	16	Fast	Low	Lossy
MinicapStream	32	Very Fast	Low	Lossy
EmulatorExtras	64	Very Fast	Low	Lossless	Only supports emulators: MuMu 12, LDPlayer 9, and AVD

Android Native

The Android native controller uses MaaAndroidNativeControlUnit for direct screenshot and input on Android.

Android Native Prerequisites

1. Android platform only

Android Native Configuration

Pass a JSON config to MaaAndroidNativeControllerCreate(config_json):

• library_path: Android native control unit library path
• screen_resolution.width / screen_resolution.height: raw screenshot resolution, also used as the touch coordinate space
• display_id: target display id, optional, defaults to 0
• force_stop: whether to force stop before start_app, optional, defaults to false

NOTE

MaaFramework converts scaled recognition coordinates back to raw screenshot coordinates in ControllerAgent. The Android native controller no longer performs any frame-to-touch remapping internally, and only clamps to screen_resolution. If the control unit reports a raw frame resolution different from screen_resolution, screencap fails immediately.

Win32

Win32 Input

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Different programs on Win32 handle input differently, so there is no universal method.

Name	API Value	Compatibility	Requires Admin	Seizes Cursor	Background	Description
Seize	1	High	No	Yes	No
SendMessage	2	Medium	Maybe	No	Yes
PostMessage	4	Medium	Maybe	No	Yes
LegacyEvent	8	Low	No	Yes	No
PostThreadMessage	16	Low	Maybe	No	Yes	Deprecated
SendMessageWithCursorPos	32	Medium	Maybe	Brief	Yes	Briefly moves cursor to target position then restores, for apps that check real cursor position
PostMessageWithCursorPos	64	Medium	Maybe	Brief	Yes	Briefly moves cursor to target position then restores, for apps that check real cursor position
SendMessageWithWindowPos	128	Medium	Maybe	No	Yes	Briefly moves window to align target with cursor then restores, does not seize mouse
PostMessageWithWindowPos	256	Medium	Maybe	No	Yes	Briefly moves window to align target with cursor then restores, does not seize mouse

NOTE

• Admin privileges mainly depend on the target program's permission level. If the target program runs as administrator, running with admin privileges is required for compatibility.
• The WithCursorPos methods briefly move the cursor to the target position, then restore it after sending the message, hence "Brief" cursor seizure, but it does not block user operations.
• The WithWindowPos methods briefly move the window so the target position aligns with the current cursor position, then restore the window position after sending the message. The cursor is not moved, so there is no mouse seizure, but the window may briefly flicker.
• Win32 also provides a Mouse Lock Follow mode: enable with MaaControllerSetOption(ctrl, MaaCtrlOption_MouseLockFollow, &enabled, sizeof(bool)) (set enabled to true to enable, false to disable). Designed for TPS/FPS games that lock the mouse to the window in the background. When enabled, the window continuously follows the mouse cursor, and RawInput counter-moves prevent the game from sensing hardware mouse movement. Use MaaControllerPostRelativeMove to inject intentional camera rotation while this mode is active. Note: On Win32, MaaControllerPostRelativeMove requires mouse-lock-follow mode to be active; calling it without this mode will fail. Only supported with MessageInput-based input methods (SendMessage / PostMessage variants).

Win32 Screencap

Reference: MaaDef.h

Combine the selected methods below using bitwise OR to provide a single value. MaaFramework will try all provided methods and select the fastest available method.

No default value. Client can choose one combination as the default.

Different programs on Win32 handle rendering differently, so there is no universal method.

Name	API Value	Speed	Compatibility	Requires Admin	Background	Description
GDI	1	Fast	Medium	No	No
FramePool	2	Very Fast	Medium	No	Yes	Available on Windows 10 1903+
DXGI_DesktopDup	4	Very Fast	Low	No	No	Desktop duplication (full-screen output copy)
DXGI_DesktopDup_Window	8	Very Fast	Low	No	No	Desktop duplication cropped to window
PrintWindow	16	Medium	Medium	No	Yes
ScreenDC	32	Fast	High	No	No

NOTE

Three predefined combination macros are provided:

• MaaWin32ScreencapMethod_All: all screencap methods
• MaaWin32ScreencapMethod_Foreground: DXGI_DesktopDup_Window | ScreenDC
• MaaWin32ScreencapMethod_Background: FramePool | PrintWindow

FramePool and PrintWindow have built-in pseudo-minimize support: when the target window is minimized, they make it transparent and click-through, then restore it without activation, allowing screencap to continue without disturbing the user.
Other screencap methods will fail when the target window is minimized. Please avoid minimizing the window when using those methods.

MacOS

The MacOS controller is used to control native macOS applications on macOS.

MacOS Prerequisites

1. macOS 14.0 or later
2. The following permissions need to be granted:
   • Screen Recording permission: Required for screenshot functionality
   • Accessibility permission: Required for input control functionality

Permission Debugging

If you encounter permission-related issues, you can reset permissions using the following commands:

# Reset screen recording permission
tccutil reset ScreenCapture

# Reset accessibility permission
tccutil reset Accessibility

TIP

After resetting permissions, you need to restart the application and re-grant the permissions.

TIP

MaaFramework is not responsible for requesting or guiding permissions; clients must implement these functionalities themselves. MaaToolkit provides a set of helper functions for this purpose; please refer to test/macos_test.

MacOS Screencap

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Name	API Value	Speed	Compatibility	Requires Permission	Background	Description
ScreenCaptureKit	1	Fast	High	Screen Recording	Yes	macOS 14.0+

MacOS Input

Reference: MaaDef.h

Select one of the values below.

No default value. Client can choose one as the default.

Name	API Value	Compatibility	Requires Permission	Background	Description
GlobalEvent	1	High	Accessibility	No	Injects into the global HID event stream via CGEventPost(kCGHIDEventTap), dispatched by the OS to the front window (automatically activates the target window)
PostToPid	2	Medium	Accessibility	Yes	Sends directly to the target process via CGEventPostToPid, no need for the target window to be in the foreground

PlayCover (macOS)

The PlayCover controller is used to control iOS applications running via fork PlayCover on macOS.

PlayCover Prerequisites

1. Install fork PlayCover on macOS
2. Target iOS app must have MaaTools feature enabled in PlayCover

Gamepad (Windows)

The Gamepad controller is used to emulate Xbox 360 or DualShock 4 gamepad input on Windows, suitable for games that require gamepad control.

Gamepad Prerequisites

ViGEm Bus Driver must be installed.

Gamepad Types

Type	API Value	Description
Xbox360	0	Microsoft Xbox 360 Controller (wired)
DualShock4	1	Sony DualShock 4 Controller (wired)

Control Mapping

The Gamepad controller uses the following mapping:

Digital Buttons (click_key/key_down/key_up)

Use MaaGamepadButton_* constants as key values. Xbox button values are used for Xbox360 gamepad, DS4 face buttons automatically map to Xbox equivalents:

Button	API Value	Xbox 360	DualShock 4 Equivalent
DPAD_UP	1	D-pad Up	D-pad Up
DPAD_DOWN	2	D-pad Down	D-pad Down
DPAD_LEFT	4	D-pad Left	D-pad Left
DPAD_RIGHT	8	D-pad Right	D-pad Right
START / OPTIONS	16	Start	Options
BACK / SHARE	32	Back	Share
LEFT_THUMB / L3	64	Left Stick Press	L3
RIGHT_THUMB / R3	128	Right Stick Press	R3
LB / L1	256	LB	L1
RB / R1	512	RB	R1
GUIDE	1024	Xbox Guide	-
A / CROSS	4096	A	✗ (Cross)
B / CIRCLE	8192	B	○ (Circle)
X / SQUARE	16384	X	□ (Square)
Y / TRIANGLE	32768	Y	△ (Triangle)
PS	65536	-	PS (DS4 only)
TOUCHPAD	131072	-	Touchpad Press (DS4 only)

Analog Input (touch_down/touch_move/touch_up)

Use the contact parameter to select the control target:

Contact	Description	x/y Range	pressure Range
0	Left Stick	-32768 ~ 32767	Ignored
1	Right Stick	-32768 ~ 32767	Ignored
2	Left Trigger (LT/L2)	Ignored	0 ~ 255
3	Right Trigger (RT/R2)	Ignored	0 ~ 255

NOTE

• ViGEm Bus Driver must be installed to use this controller.

WlRoots (Linux)

The WlRoots controller is used to control applications running in a wlroots compositor on Linux.

Compositor Requirements

The wlroots compositor to be controlled must support the following protocols:

• Wayland Core Protocols
• virtual-keyboard-unstable-v1
• wlr-screencopy-unstable-v1
• wlr-virtual-pointer-unstable-v1

NOTE

It is recommended to start a nested compositor session to use this controller. Controlling the compositor currently used by the desktop is not recommended, as this may lead to unexpected behavior during task execution.

Keyboard input notice

Keycodes for MaaControllerPostKey{Down,Up} in the WlRoots controller are by default evdev key codes, defined in linux/input-event-codes.h.

If you prefer using Win32 Virtual-Key codes (VK_*), pass true for the use_win32_vk_code parameter when calling MaaWlRootsControllerCreate. When enabled, key codes passed to MaaControllerPostClickKey / MaaControllerPostKey{Down,Up} are interpreted as Win32 VK codes and translated to evdev codes internally. This flag is decided at creation time and cannot be changed at runtime; it makes it easy to reuse key handling logic written for the Win32 controller.