Your phone can now get an AI helper that looks at the screen and presses the buttons for you, and the whole thing runs on the handset. Opposable, an Android app, published the details on its blog on 21 October 2026, in a post credited to Reval Labs.

The app reads what is on screen through Android’s accessibility service, the same hook that screen readers use. It can then tap and type inside any other app. Opposable says Claude, ChatGPT or Codex can steer it from a computer. When none of those is connected, a small model takes over and does the job locally, with no server involved.

The model is called GUI-Owl 1.5, and the post credits Alibaba’s Mobile-Agent team with building it. Technically it is a fine-tune of Qwen3-VL (Alibaba’s vision-language model) with 2 billion parameters. Given a screenshot, it answers with one action, such as “Click the Files app icon.” Opposable squeezed it to 4 bits per weight and runs it through llama.cpp, the open-source engine for local models. The language part weighs 1.05 GB and the vision part 445 MB, and users download about 1.5 GB from the app’s settings.

Speed is the catch. Each step has three stages: turning the screenshot into tokens, reading them alongside the task, and writing the action. Opposable’s first version took 10.6 seconds per step on the CPU of a Mac M2 laptop. After trimming the top strip, bottom buttons and on-screen keyboard from each screenshot, discarding tokens for blank background, caching the long system prompt, and skipping the coordinates whenever the model’s sentence points at a single element on screen, a step dropped to 7.3 seconds. Accuracy held at 27 of 36 screens on Opposable’s own graded set. At that pace a ten-step errand would take roughly 73 seconds on the laptop, which is slow next to a person but tolerable for something running in the background.

The phone’s own chip does better on one stage. Opposable exported the vision encoder and had Qualcomm AI Hub compile it for the Galaxy S25’s neural processor. In Qualcomm’s device lab it ran in 236 milliseconds, against 3.9 seconds on the M2, about 16 times faster. A cheaper 8-bit version was slightly quicker at 213 milliseconds but its answers strayed from the reference, so the team kept the full-precision one. The first load takes 72 seconds while the chip prepares, then 0.39 seconds afterward. The post gives no end-to-end step time measured on an actual phone, so the laptop figure is the only full-step number.

Reliability needed its own fix. Opposable also tried eight tasks the model was never tuned for. Handing over the whole job worked 4 times in one round and 3 in the next. Splitting it into small goals, and confirming each one against the screen’s element list before moving on, worked 6 times out of 8 in both rounds. These are the team’s own tests on an emulator, and the sample is tiny.

The privacy angle is plain: with the model on the handset, what appears on your screen is not sent to a remote model to decide where to tap. That matters because a screen-reading agent sees messages, banking apps and everything else. Opposable also runs about forty common jobs, like setting an alarm or switching on Wi-Fi, as ordinary scripted code that needs no AI.

The limits are stated openly. Only Snapdragon phones have the chip-accelerated build so far, and Pixel, MediaTek and Exynos phones fall back to the graphics chip or the CPU. On apps it has not seen, the small model is far weaker than a big hosted one, so the app passes those jobs to a connected Claude, ChatGPT or Codex.

For anyone building phone assistants, the open question is whether a step time near seven seconds drops far enough once the full loop runs on phone silicon, because that figure decides whether on-device agents stay a privacy option or become the default.

Reported by Opposable (the Opposable blog, by Reval Labs) on 21 October 2026.