JournalAI Engineering

Field guide / 6

Android repositories accepted 71% of studied AI pull requests. iOS accepted 63%.

A study of 2,901 agent-authored pull requests suggests that routine mobile work is a better fit for autonomous contributions than structural changes.

Sep 13, 20266By ISH Team
Android repositories accepted 71% of studied AI pull requests. iOS accepted 63%.
Advertisement

Android repositories accepted 71% of studied AI pull requests. iOS accepted 63%.

The useful question about AI-authored pull requests is rarely whether an agent can produce code. Maintainers have to decide whether a change belongs in a real project, whether it respects the platform's conventions, and whether the review time is justified. A study of open-source mobile repositories puts some numbers around that decision.

Researchers studying AI coding-agent adoption in Android and iOS analyzed 2,901 agent-authored pull requests from 193 verified open-source repositories in the AIDev dataset. Android projects received roughly twice as many of the studied pull requests and accepted 71% of them. iOS projects accepted 63%. The paper also found meaningful variation by agent on Android, while the iOS results did not show the same agent-level difference.

The figures are not a measure of which platform is better for AI, nor do they prove that an accepted pull request is correct or valuable over time. They describe a curated research sample and review decisions made in particular projects. Still, they are a useful counterweight to benchmark scores. Code enters production through a social and technical process, and the process has different friction on different platforms.

A merged pull request is a richer signal than generated code

An agent can complete a task locally and still send a poor pull request. It may change an API that maintainers were trying to retire, miss a project convention, add an unnecessary dependency, or solve a visible symptom while leaving the underlying problem. Merge decisions fold in those concerns.

The study's results show that task type mattered too. Routine tasks such as features, fixes, and UI work had the highest acceptance rates, while refactor and build changes had lower success and longer resolution times. This is a pattern worth taking seriously because structural work asks an agent to infer more of the project's unstated history. A UI adjustment can often be checked against a visible result. A build or refactor change can alter ownership, tooling, dependencies, and future maintenance cost at once.

That does not mean agents should never touch infrastructure. It means maintainers should give structural work more context and a more demanding acceptance path. Require an explanation of dependency changes, a before-and-after build result, rollback notes, and tests that demonstrate the intended boundary. Treat a generated refactor as a proposal to inspect, not a mechanical cleanup to merge because the diff is neat.

Platform differences are workflow clues

The Android and iOS difference should not be turned into a story about one community being more willing to accept automation. The paper notes that Android had more observed agent pull requests and that acceptance varied by agent there; the sample and repository selection shape those outcomes. iOS projects also have their own languages, build systems, release workflows, and design expectations.

The useful operational reading is narrower. Platform-aware evaluation matters. A team that measures an agent only on generic coding puzzles will miss the conventions that reviewers actually apply. For mobile work, that can include manifest or project-file changes, dependency management, accessibility behavior, visual regressions, simulator or device testing, and release tooling.

Write those expectations down where an agent can use them. A short repository guide can say which generated files should not be edited, which commands count as a successful build, which screenshots or tests a UI change needs, and when a change must be split into a separate pull request. This also helps human contributors. The difference is that an agent may otherwise confidently fill an information gap with an irreversible edit.

Start where review has a clear oracle

Routine work is a good place to establish trust because the review can have an observable oracle. A localization change can be checked against the requested strings. A bug fix can come with a reproducer that failed before the patch and passes afterward. A narrow UI change can include an accessibility assertion or screenshot comparison. These are not substitutes for review, but they give the reviewer a concrete way to evaluate the agent's claim.

The AIDev dataset exists precisely because activity in real repositories reveals more than a synthetic task can. Its initial release aggregated 932,791 agent-authored pull requests across 116,211 repositories and five named coding agents, with a curated subset containing richer review and timeline information. The mobile study draws a much smaller, platform-specific sample from this wider record.

That scale is valuable, but it has limits. Agent attribution in public pull requests can be imperfect. Acceptance is affected by repository health, maintainer availability, contributor relationships, and the way a task is categorized. The study does not establish causation between an operating system and a merge decision. It identifies an empirical difference worth investigating when teams plan where to deploy agents.

Give agents a path to earn broader scope

The easiest way to create review fatigue is to hand an agent a broad issue and ask it to “fix everything.” A better path starts with small, independently checkable contributions. Track which kinds of changes merge without extensive revision, which tests catch mistakes, and which files create recurring reviewer questions. Use that record to shape the next task.

For higher-risk work, separate discovery from modification. Ask the agent to map the build or module boundary first. Have it identify likely affected files and explain why. Review that plan before it edits. After the change, ask for a compact verification report that links each requirement to a command, test, or observed result. This keeps a pull request readable even when the implementation was generated quickly.

The mobile study does not offer a universal acceptance formula. It does show that the unit of adoption is not “the coding agent.” It is a particular task, on a particular platform, in a repository with its own review norms. Teams get more useful automation when they design around that fact.

Sources

#AI coding agents#open source#Android#iOS#pull requests#software engineering
Advertisement

Keep reading

Related stories

Browse the archive