GitHub's Issue Arborist reports the links it refuses to make
An agent that organizes a busy issue tracker can turn one plausible guess into permanent structure. Link unrelated bugs under the same parent and maintainers may start treating them as one workstream. Miss a real cluster and the backlog stays messy, but the agent has not damaged it.
GitHub's open-source Issue Arborist workflow is designed around that imbalance. It scans recent issues each day and creates parent-child relationships. It also records the relationships it noticed but chose not to create.
GitHub's 31 August account of the workflow says the previous three runs made 129 link_sub_issue calls and opened one new parent issue. The report also names clusters left for manual review because textual similarity did not establish a hierarchy. Acting, abstaining, and leaving evidence for either decision is more useful than the activity count by itself.
Similar titles do not establish hierarchy
Five related reports may share one root cause. They could instead be duplicates, separate symptoms, competing implementations, or repeated failures from different runs. A model can cluster the language without knowing whether one issue is broader than the others.
Issue Arborist's workflow source distinguishes those questions. It searches for features with implementation tasks, epics, root-cause relationships, tracking issues, semantic similarity, and orphan clusters. Before making a link, the agent must decide whether the proposed parent is broader and whether the connection improves traceability.
The prompt instructs the agent to prefer precision over recall. If it is unsure, it should leave the issues alone. That rule matches the cost of the operation: unlinking is manual, while a missed relationship can be reviewed on the next daily run.
A false positive changes the shared backlog. A false negative preserves the current state. The automatic-link threshold should be high because those outcomes do not cost the same amount to repair.
Limits contain a poor run
The GitHub workflow downloads the latest 100 open issues without parents. Each record contains a number, title, URL, timestamps, labels, and the first 800 normalized characters of the body. One run can create at most five parent issues and 50 sub-issue links. Creating a new parent requires at least five strongly related orphan issues.
The finite input, mutation caps, and minimum cluster size do not guarantee correct judgment. They do limit the amount of cleanup a bad run can create.
That containment matters even when a team trusts the model. Issue relationships can change while a run is in progress, and a reasonable classification can still be stale by execution time. A bounded batch gives the executor fewer assumptions to recheck and gives maintainers a manageable set of changes to inspect afterward.
The model does not receive an unrestricted write token and compose arbitrary GitHub API requests. It proposes specific safe outputs for creating an issue, linking a sub-issue, or creating a discussion. The project's safe-output specification defines the validation and limits applied by the executor. The proposal and the mutation are separate stages.
That boundary was active during the 31 August run. Its public Actions record contains ten warnings where the safe-output layer skipped proposed links because those issues already belonged to the stated parents. The workflow still completed successfully. The agent suggested redundant work; the executor checked live state and declined it.
The 129 link_sub_issue calls reported across three runs therefore should not be read as 129 distinct, verified improvements. Tool calls can include proposals that a deterministic layer rejects.
Record the near misses
Agent dashboards commonly count completed actions, tokens, latency, and errors. A triage agent also needs to expose the decisions maintainers may want to challenge.
Issue Arborist publishes a temporary discussion after each run. The report template covers parents created, links made, reasoning, observations, and "Potential Relationships (For Manual Review)." A 29 August report shows the format with real output: 100 issues analyzed, one new Deep Report parent, several linked clusters, and a note that other candidates were too broad, isolated, or already organized.
Maintainers can use that record to catch an obvious missed relationship. It also reveals when the confidence threshold is too strict and distinguishes a deliberate refusal from a failed workflow. Repeatedly rejected clusters become visible instead of being reconsidered in silence every morning.
The public report does not prove every accepted link was correct. GitHub is describing its own system, and the available activity count is not an independent review. A useful evaluation would sample accepted links and rejected candidates, then check later human reversals and relationships that maintainers eventually added.
Design the abstention path before choosing a model
The same problem appears in CRM deduplication, incident grouping, document filing, support routing, and agent memory. Each system turns a model's interpretation into shared structure. Before choosing a model, a team should answer:
- What evidence permits an automatic mutation?
- Is a wrong action more expensive than a missed one?
- Which limits contain a bad run?
- Where will plausible rejected actions remain visible?
Inputs require their own protection. Public issue bodies are untrusted text, as our IssueTrojanBench guide explains. A strict linking threshold does not prevent prompt injection. The workflow still needs constrained tools, sanitized outputs, and permissions limited to its narrow job.
A model gateway such as api.ish.chat can support comparisons across providers, but the reliable behavior belongs in the surrounding application. Limits, current-state checks, and idempotency should hold regardless of which model proposed the link.
Issue Arborist treats "no" as an output that maintainers deserve to see. Its refusal log tells them which possible structure the agent considered, why it left the backlog unchanged, and where a human decision could still help.
