AI now handles smartphone tasks for users who’d rather not move their fingers
A new tool called Opposable allows AI agents to control smartphones by tapping, swiping, and typing on screens via a connected computer. It uses visual processing through a vision-language model and accessibility services on Android, while iPhones require a companion device. The system charges users for its services, raising concerns about efficiency and user dependency.
Opposable leverages a vision-language model (GUI-Owl 1.5 2B) via llama.cpp to interpret smartphone screens and execute user-directed tasks, with Android support using accessibility services and visual input from screenshots. While some common actions like setting alarms or toggling Wi-Fi are handled by pre-programmed code for efficiency, more complex tasks rely on AI inference, resulting in high latency—up to 96 seconds for basic app navigation. The system requires a connected computer for iPhones, with Mac and Windows options available, and charges users per interaction, with costs reaching $0.44 for simple queries.
This tool may accelerate the normalization of AI-mediated digital dependency, particularly among users prioritizing convenience over engagement. While it could benefit individuals with physical disabilities, its high cost and inefficiency may exacerbate digital inequity and encourage over-reliance on AI for trivial tasks. The broader societal impact remains uncertain, but such tools may reshape expectations around personal agency in device use.