GitHub Copilot metrics now break out agent apps
Teams that run more than one coding agent have had an awkward measurement problem. A single Copilot usage total can show that activity happened, but not whether it came from Copilot coding agent or a connected agent app. That makes a rollout hard to evaluate. A higher total might mean a new tool is being used, or it might simply reflect work that moved from one agent to another.
GitHub now exposes agent-app activity as a separate breakdown in its Copilot usage metrics reports. The change is useful, but it is easy to turn a few counters into a story they cannot support. The practical work is defining the question first, pulling the right report, keeping the count types separate, and checking the result against delivery evidence.
What the new field measures
GitHub added an optional totals_by_3rd_party_agent array to its one-day and 28-day Copilot metrics reports for enterprise, organization, enterprise-user, and organization-user views. The array has one record for each recognized agent app. Each record includes:
agent_name, a display name that can change.agent_id, the stable value to use when grouping data across reporting periods.user_initiated_interaction_count, the count of jobs users started with that agent app.session_count, available only in aggregated enterprise and organization reports.
The update separates activity by agent. It does not make all agent work directly comparable. A job start is not a completed pull request, a merged change, a review comment, or a developer's time saved. A session count is not a user count either. Use these fields as adoption and workflow-volume signals, then compare them with the engineering outcome you care about.
GitHub says the field is optional. Its absence can mean there was no recognized agent-app activity in the report period, rather than a zero that should be added to every chart. Activity from agents GitHub cannot identify is omitted. Keep that limitation visible when you describe coverage.
Start with one decision
Write down the decision the report should support before making a dashboard. For example:
- Should we keep the pilot open for another month?
- Did the migration change which agent engineers start jobs with?
- Are teams using the agent connected to our repository workflow?
Then choose a small set of measures. For a pilot, weekly starts by agent_id, active repositories, merged pull requests, and review findings that escaped into production may be enough. The last three do not come from this Copilot metric. They come from your GitHub data, CI system, and incident process.
Keep the unit next to every chart. An axis called "adoption" can conceal whether it shows jobs, sessions, people, or seats. If you use both starts and sessions, label them separately. GitHub explicitly warns that the nested agent-app user_initiated_interaction_count is different from the top-level field with the same name. Do not add the two together.
Fetch the report, then preserve the raw file
The metrics API returns signed download links, not the report rows themselves. For an organization, the one-day endpoint is:
curl -L \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-H "X-GitHub-Api-Version: 2026-03-10" \
"https://api.github.com/orgs/ORG/copilot/metrics/reports/organization-1-day?day=YYYY-MM-DD"
The response contains download_links and report_day. The 28-day organization endpoint is /orgs/ORG/copilot/metrics/reports/organization-28-day/latest; it returns links plus the reporting range. GitHub documents equivalent enterprise and user-report endpoints.
Store each downloaded file with the retrieval time and report date. Signed URLs have a limited lifetime, and a report can be split across multiple files. Keeping the raw NDJSON files lets you rerun a parser when GitHub adds a field or when your own aggregation changes. Do not commit these files to a repository if they contain user-level data.
Access is intentionally narrow. Organization reports require an organization owner or an authorized user, and fine-grained tokens need the read-only "Organization Copilot metrics" permission. Enterprise reports have their own read permission. Use a reporting identity with the smallest scope that can fetch the intended view, and put its token in your existing secret store rather than in a dashboard script.
Group on agent_id, not the display name
The safest first aggregation is short:
report day + agent_id -> job starts
report day + agent_id -> sessions, when present
Keep agent_name as a label only. GitHub says display names can change, so grouping on them can split a single agent into two apparent series or join unrelated data after a rename. Maintain a small lookup table for a friendly report label, the stable ID, rollout date, and any known coverage limitation.
Calculate a baseline before you announce a result. Compare the same number of complete days before and after the rollout. Avoid comparing a partial current day with a finished prior day, or an individual report with an aggregated report. If usage fluctuates with release weeks, incidents, or holidays, annotate those periods instead of calling every movement an agent effect.
The Git worktrees playbook is relevant here. Parallel agent tasks can produce more sessions while changing little about completed work. A worktree is a useful isolation mechanism, but session volume alone cannot tell you whether the resulting patches were reviewable or correct.
Join metrics to delivery data without pretending it is causation
Agent metrics become more useful when joined to a shared reporting grain, such as organization and week. Create a table that has one row per week, agent_id, then add the engineering data your team already trusts:
| Measure | What it can answer | What it cannot answer alone |
|---|---|---|
| Agent-app job starts | Which agent people choose to start | Whether a job was useful |
| Agent-app sessions | How much aggregated agent activity ran | How many people used it |
| Merged pull requests | Whether delivery volume changed | Who or what caused the change |
| CI failures or reverts | Whether quality signals moved | Whether one agent caused the change |
Look for directional evidence, not a manufactured score. A pilot may be healthy if a stable group keeps using an agent while review time and rollback rates stay within the team's normal range. A sudden jump in starts accompanied by more abandoned pull requests is a reason to inspect prompts, repository setup, or agent permissions before expanding access.
Avoid user-level reporting by default. The user reports can help an authorized administrator diagnose enablement problems, but a public team scoreboard changes the incentive from good engineering to visible activity. Aggregate results for routine reporting and use individual data only for a specific support or governance need with clear access controls.
Check what the agent could actually use
Usage counts do not show the context an agent had. GitHub Copilot code review can use repository skills and MCP servers, while MCP calls for code review are limited to read-only access. If one team has a well-maintained .github/skills directory and another does not, the same agent label can cover different operating conditions.
Record important configuration changes beside the rollout timeline: new skills, enabled MCP servers, model changes, permission changes, and CI changes. The Agent Plugins guide explains a related portability boundary: a packaged workflow can load in multiple clients without providing an identical experience in each one.
Fetch the completed report on a fixed schedule, validate the schema, group by stable agent ID, join a few delivery and quality measures, and review unusual changes with the teams involved. The report gives you a better starting point for that conversation. It does not replace code review, test results, or engineering judgment.
