JournalResearch

Field guide / 5 min read

One person handles most agentic PR oversight in a 2,361-repository study

A 25,264-PR study found oversight usually fell to one developer. Teams should measure review queues, rework, ownership, and approval capacity.

Aug 25, 20265 min readBy ISH Team
One person handles most agentic PR oversight in a 2,361-repository study
Advertisement

One person handles most agentic PR oversight in a 2,361-repository study

A coding agent can open pull requests faster than a maintainer can read them. Dashboards count the patches. The review time usually remains a human cost.

A July 2026 study examined 25,264 merged or closed agentic pull requests across 2,361 popular GitHub repositories. More than two thirds involved a single developer who both reviewed and modified the agent's contribution. Work involving several humans was uncommon.

Researchers Maliha Noushin Raida and Daqing Hou measured adoption, PR activity, and collaboration patterns. They did not measure burnout, correctness, security, or total review effort, so their results cannot show that agents reduce teamwork or harm code quality. They do show where oversight sat in this sample: usually with one person.

What was measured

The paper used the AIDev-pop dataset and focused on pull requests from GitHub Copilot, OpenAI Codex, and Claude Code. Its observation window covered May, June, and July 2025. The curated repositories had more than 100 GitHub stars, and the analysis included merged and closed pull requests.

Adoption was uneven. The median repository recorded only one or two agentic pull requests during those three months. Intensive use appeared in a small set of projects. Projects with one to five contributors had higher participation ratios and more agentic PR activity on average than medium and large projects.

The authors also calculated agentic PR productivity as pull requests per human participant. That measure ignores PR size, complexity, quality, and review effort. Many tiny documentation changes can produce a higher count than a few difficult fixes without saving more human time.

The sample covers popular public repositories during an early adoption period. It may not describe private company codebases, less popular open-source projects, or teams that use agents through workflows the dataset could not observe.

Patch throughput is not review throughput

Agents can work concurrently and return another patch as soon as a task is assigned. A reviewer still has to understand the issue, inspect the diff, check the tests, find missing requirements, and decide whether the change belongs in the project.

When one developer both reviews and modifies an agentic PR, that person may be diagnosing the patch, correcting it, and deciding whether it can merge. The PR count records one agent contribution without recording how much human repair made it acceptable.

This is the missing half of many per-agent usage metrics. Activity counts need context from review wait time, human follow-up commits, requested changes, reverts, and incidents linked to agent-authored code.

One-person review can be reasonable for low-risk maintenance in a small project. Repositories still need an explicit point at which a second person joins the review. That threshold may depend on the files touched, the permissions involved, or the cost of a wrong change.

Preserve an independent approval path

GitHub applies a useful separation to Copilot pull requests. If a repository requires approvals, the approval of the person who assigned the task to Copilot does not count toward the required number; someone else must approve before merge. Copilot code review leaves comments rather than an approval or request-changes verdict, so it does not satisfy a required approval.

The same principle can govern other agent tools. Machine feedback can help find defects, but it should not silently replace a human approval required by repository policy.

Path-based ownership identifies the people who should inspect sensitive changes. GitHub's CODEOWNERS file can request reviews from the owners of matching files, and protected-branch rules can require code-owner approval. Authentication, billing, database migrations, deployment workflows, security configuration, and public API contracts can then follow a stricter path than formatting or documentation.

Repository instructions solve a different problem. An AGENTS.md file can give the agent test commands, architectural boundaries, and forbidden actions. Review policy still has to name who may merge and which changes require another person.

Count the queue around the agent

An agent-PR dashboard should include the work that lands on reviewers:

  • agentic PRs opened, merged, closed, and reverted;
  • time to first human review and time to merge;
  • human follow-up commits and review rounds;
  • changed files, ownership boundaries, and risk class;
  • CI evidence, security checks, and missing tests;
  • reviewer concentration, including how much agent work reaches each person.

Reviewer concentration exposes a bottleneck that PR totals hide. Ten simultaneous agent tasks can create ten review obligations for one maintainer. A lower concurrency limit may improve end-to-end throughput when it keeps the queue within the team's capacity.

The pull request should also contain the evidence needed for review: the issue, plan, diff, commands run, test results, and unresolved uncertainty. This follows the same logic as trajectory-based agent evaluation. A patch is easier to judge when the route to it is visible.

A small policy that scales

Teams can start with five rules.

  1. Label agent-authored pull requests consistently, regardless of vendor.
  2. Assign an accountable human before the agent begins work.
  3. Require an independent human approval for privileged or high-risk paths.
  4. Cap concurrent agent tasks according to available review capacity.
  5. Revisit the limit when review time, rework, or reverts rise.

In the study, one-person oversight was already the dominant pattern while intensive agent adoption remained uncommon. If the volume of generated patches rises, teams will need to budget for the informed human attention that turns those patches into maintainable code.

Primary sources

#coding agents#pull requests#code review#open source#developer workflow
Advertisement

Keep reading

Related stories

Browse the archive