AutoMobile & Mobile Use
Writing about what I've been up with AutoMobile project, why I want to keep working on it
The best of mobile tooling has been inaccessible behind walls of cost and specialized expertise. There is no cohesive UX where we tie them together, its more like AutoCad 2008 with tons of options that take a really long time to work together. Automating any part of tedium in mobile engineering has therefore been out of reach.
I built AutoMobile to change that. It is an open source MCP server that lets AI agents control Android and iOS devices using natural language. Most features don't require an SDK dependency in your app. The rest of this blog post is to explain the motivations, roughly how it works, and where I want to take this project. If you just want to kick the tires skip to the end.
Mobile Use
2025 kicked off with browser use by leveraging the pre-existing Playwright and Model Context Protocol (MCP). This granted not just the ability to write AI driven tests to a wider audience but also the ability to explore, debug, prototype. We saw both an explosion of vibe coding as well as a ton of companies — but not as much for mobile.
What did the web folks have that mobile didn't? A frontend engineer could ask an AI agent to inspect a page, click through a user flow, check local storage, profile performance, and file a bug — all in one session. A lot of mobile engineers were still alt-tabbing between Cursor and Android Studio or XCode.
In the meantime I was attempting to improve UI testing and had started AutoMobile with that goal in mind. But as I iterated I noticed a number of different solutions for mobile engineers started coming out with different technical implementations and goals — and even more in the past week! I came to my own conclusion that solving UI testing tooling problems opens up full mobile use. Which brings us to where the project is today.
What AutoMobile Does
AutoMobile is an MCP server that lets AI agents control Android and iOS devices using natural language. You describe what you want to do, and the AI handles the implementation details. Here's what that looks like in practice:
Explore the app and explain how it works. AutoMobile's MCP tools are built to be used by AI agents to navigate and inspect mobile devices similar to how real users do. This means agents can discover features through well designed UX and be hindered by inaccessible elements. It is no replacement for real user UX research, but it can create a helpful early development feedback cycle.
Reproduce a bug from a crash report. Paste a bug report into your AI agent and say "reproduce this." AutoMobile will attempt to reproduce and highlight the issue observed in an automatic video recording and device snapshot. The MCP server can provide the time series performance data and logs to be correlated with each action taken.
Profile app performance. What if you could have a constant indicator of performance always on for debug and release versions of your app? Write what you want to measure and AutoMobile captures timing data while displaying real time numbers in its IDE companion plugin. These are microbenchmarks to be sure, but we should be able to use this data to make performance work more accessible.
All of this works because the element search is designed to be highly reproducible. AutoMobile automatically determines which element and where on the screen the intended interaction should be taken. This is how it continues to work when the design changes slightly or the layout renders differently on a different device — without relying on AI for every interaction. This is how it can operate a multi-touch pinch-to-zoom with rotation on Google Maps without the touch being captured by the bottom sheet.
Its also how I made a pretty neat IDE plugin:
State of the Project
Android platform support is pretty solid. You can see the various recorded examples I have on the AutoMobile GitHub Pages site and those should Just Work. iOS I've yet to make recordings for but I'm able to run AI agent & IDE plugin on iOS simulators on any of Apple's built-in apps.
What's Next
This isn't done - I'm actively building AutoMobile and want to get it to a 1.0 release. There is no specific timeline for that, but I've put up design docs to organize the project vision and architecture. I'll be writing about the technical journey—performance improvements, how it reuses its observability, specific challenges of emulator/simulator operation, and keeping a mono-repo with big ambitions organized. As I get ideas or feature requests I'll add them. And some stand-alone desktop apps are definitely in the works.
If you're a mobile engineer frustrated by the gap between what your tooling could be and what it actually is, give AutoMobile a try. Check out the recorded examples on the AutoMobile GitHub Pages site, and if you want to talk about it you can find me on Bluesky or DM me in any Slack community we share.
Install & Go
curl -fsSL https://raw.githubusercontent.com/kaeawc/auto-mobile/main/scripts/install.sh | bashRun to setup AutoMobile (with interactive prompts before actions)
