Source Reading Methodology tests the quote and the citation you deleted
A line-numbered code block can still be wrong. The path may name the wrong file. A symbol may be off by one word. Six inconvenient lines may have vanished from the middle. Most readers will not reopen the repository and compare every byte.
The open-source Source Reading Methodology turns that comparison into an engineering task. Every technical claim should lead to a specific line in a pinned source tree, and scripts should check the citations. The workflow was designed for courses and books about large repositories, but its core checks also fit architecture articles, migration guides, and code review explanations.
A line number needs a source version
The skill instructions begin by recording a commit and creating a tag for the repository under study. That version appears in every downstream document. Without it, line references drift, and a reader cannot distinguish an original mistake from a later upstream edit.
We reviewed commit f45f57988f58d45a7eab9b5b736e3024f3c5efbb, dated August 25, 2026. The figures and file behavior in this article refer to that snapshot.
Each cited code block carries a starting line, ending line, and path relative to the repository root. Outline anchors are either verified or candidates. Even a verified anchor must be reopened during drafting because the useful excerpt may sit beside the originally noted line.
For a short article, this discipline needs only an immutable commit, exact locations for consequential claims, and clear labels for inference. If the implementation cannot be found, the method says to publish the search terms instead of filling the gap with a familiar pattern.
Byte matching catches tiny changes
Quoted code must match the source byte for byte. Indentation, attributes, comments, blank lines, and punctuation remain intact. A long excerpt may skip material only through an explicit comment containing ....
The repository's 29-item pitfall log records a silent six-line deletion from a quoted comment, SIGSYS_CODE changed to SIGSYS, a moved semicolon, and a missing #[test] attribute. None looks dramatic on the page. Each weakens the evidence.
Its recommended verifier splits an excerpt at explicit omission markers and searches the source for each continuous segment. A segment that cannot be reconstructed receives a FABRICATION result. Correct text with incorrect line numbers can be repaired automatically.
The omission rule needs its own tests. One validator treated any comment containing ... as an omission. Ordinary documentation containing an ellipsis then became an unchecked hole. The project narrowed the marker to a comment whose content begins with ..., and added a warning when an excerpt's length does not match its declared range.
Byte matching cannot establish that the path is correct if the verifier searches a whole corpus for matching text. The workflow therefore calls for a separate check that the named file exists and the line range fits it.
Validation can reward deleted evidence
One draft in the pitfall log started with 56 references and several validation errors. The delivered version had 22 references and no errors. Deleting the citations made the validator green.
The proposed response is a citation-density floor plus a comparison of reference counts across revisions. Omission density is tracked too, because a writer can compress difficult excerpts into legal gaps. These checks cannot prove that every necessary citation exists. They expose two easy ways to lower the amount of evidence without fixing it.
Benchmark publishing has the same completeness problem. A runner may validate every file it receives while remaining unaware of unpublished tasks. Our FuXi artifact review found public scenarios, runners, and result summaries ending at different task counts. A validator needs an expected inventory as well as rules for the items that are present.
Source and interpretation need different labels
The full four-stage method pins the source corpus, builds an outline around one question, produces chapters with exact anchors, and compiles them into an HTML book. It recommends at least one comparison repository so a writer does not present one project's choice as the only possible design.
Code lines can show what an implementation does. They do not establish why the authors chose it, whether a comment matches runtime behavior, or whether the result is correct. The method looks to repository instructions, module documentation, tests, and official material for design intent. Writers must label inference and identify comments as comments.
The same distinction applies to AGENTS.md. Repository instructions state expected practice; code and tests reveal enforcement. Our AGENTS.md design guide separates guidance from policy.
The lightweight version
An ordinary documentation pipeline does not need the eight-part chapter format or book builder. It can adopt six checks:
- Pin the repository commit used for research.
- Attach each consequential implementation claim to an exact path and line range.
- Copy excerpts directly and mark every omission.
- Match bytes, then validate paths and ranges separately.
- Keep a minimum citation count so deleting evidence cannot fix a failure.
- Label inference, comments, and missing implementations.
The project also translates prose rules into regular expressions. Its 87-line style scanner strips code blocks, inline code, and direct quotations before scanning reader-facing prose. We ran it across the three sample chapters and received zero matches. The script cannot judge an explanation's quality. It can prevent a known banned construction from returning unnoticed.
When comparing models through ISH chat or an API harness, keep the commit, source corpus, prompt, and verification rules constant. Add a completeness check as well. Byte-perfect citations are still incomplete if the difficult evidence disappeared before validation began.
