How can I reliably simulate touch events on Android without root (like Automate and Tasker)?



How can I reliably simulate touch events on Android (without rooting) from Java outside my app which runs as a background service?

While this question has been asked before, most answers utilise ADB. (such as How to simulate touch events on Android device?)

https://github.com/chetbox/android-mouse-cursor offers a good solution using Accessibility, but is not very reliable as not all views respond to it, and games do not respond at all most of the time.

private void click() {
  AccessibilityNodeInfo nodeInfo = getRootInActiveWindow();
  if (nodeInfo == null) return;

  AccessibilityNodeInfo nearestNodeToMouse = findSmallestNodeAtPoint(nodeInfo, cursorLayout.x, cursorLayout.y + 50);

  if (nearestNodeToMouse != null) {
    logNodeHierachy(nearestNodeToMouse, 0);
    nearestNodeToMouse.performAction(AccessibilityNodeInfo.ACTION_CLICK);
  }

  nodeInfo.recycle();
}

This is the current code used by https://github.com/chetbox/android-mouse-cursor.

Android Version is 8.0, stock Android

Is there a better, more reliable way to simulate these touch events from Java? Thanks in advance!

Answer

As suggested, the best way to simulate touch events since Nougat (API 24) is by using an accessibility service and the AccessibilityService#dispatchGesture method.

Here is how I did to simulate a single tap event.

// (x, y) in screen coordinates
private static GestureDescription createClick(float x, float y) {
    // for a single tap a duration of 1 ms is enough
    final int DURATION = 1;

    Path clickPath = new Path();
    clickPath.moveTo(x, y);
    GestureDescription.StrokeDescription clickStroke =
            new GestureDescription.StrokeDescription(clickPath, 0, DURATION);
    GestureDescription.Builder clickBuilder = new GestureDescription.Builder();
    clickBuilder.addStroke(clickStroke);
    return clickBuilder.build();
}

// callback invoked either when the gesture has been completed or cancelled
callback = new AccessibilityService.GestureResultCallback() {
    @Override
    public void onCompleted(GestureDescription gestureDescription) {
        super.onCompleted(gestureDescription);
        Log.d(TAG, "gesture completed");
    }

    @Override
    public void onCancelled(GestureDescription gestureDescription) {
        super.onCancelled(gestureDescription);
        Log.d(TAG, "gesture cancelled");
    }
};

// accessibilityService: contains a reference to an accessibility service
// callback: can be null if you don't care about gesture termination
boolean result = accessibilityService.dispatchGesture(createClick(x, y), callback, null);
Log.d(TAG, "Gesture dispatched? " + result);

To perform other gestures, you might find useful the code used for testing the AccessibilityService#dispatchGesture implementation.

EDIT: I link a post in my blog with an introduction to Android accessibility services.



Source: stackoverflow