INDRA cut misgrounded quotes from 9.2% to 0.6% under heavy archive loads
An archive can be public and still be practically unreadable. Five decades of litigation have released millions of internal memos, reports, emails, advertisements, and depositions from tobacco, fossil-fuel, chemical, pharmaceutical, food, and firearms companies. The UCSF Industry Documents Library alone holds roughly 150 million pages. Search can find a memo. It cannot read enough of that record to trace a strategy across several industries.
INDRA, a new research platform from Stanford historians Daniel Akselrad and Robert N. Proctor, joins several of those collections and gives an LLM access to a researcher-selected subset. A chatbot over documents is familiar. INDRA is more interesting for the restrictions around its model: a closed evidence corpus, sentence-level provenance labels, a hard context limit, and deterministic code that checks quotations before the user sees them.
The authors' early tests report a large difference. Under heavy document loads, an off-the-shelf Claude Sonnet 5 attributed 9.2% of real quotations to the wrong document. INDRA reported 0.6%. Across 3,643 quotations tested under two loads, INDRA fabricated none. The platform's own authors ran these tests, and the result has not been independently reproduced. Its controls can still be evaluated separately from that result.
A reading room with a locked door
INDRA federates sources including the UCSF Industry Documents Library, ToxicDocs, and Stanford's Research Into the Impact of Tobacco Advertising archive. ToxicDocs describes its collection as millions of pages of previously secret material released through toxic-tort litigation, including internal memoranda, board minutes, unpublished studies, and expert reports.
Researchers first search across the collections, then choose the documents that enter an evidentiary sandbox. Once a session starts, the model cannot fetch a new web page or silently add another source. INDRA gives it OCR text from the selected documents and caps document context at 500,000 tokens, with another 50,000 reserved for instructions and conversation.
That ceiling runs against the sales pitch for ever-longer context windows. INDRA treats it as quality control. The authors say the bare model's quote accuracy fell from 96% to 61% as its context filled to one million tokens. At INDRA's 500,000-token cap, every retrieved quotation in their test was exact and correctly attributed. Documents larger than the cap are sliced deterministically into overlapping chunks.
Systems built through api.ish.chat can apply the same rule: limit retrieval, reserve context for the exchange, and decline to answer when the selected evidence cannot support a claim.
Provenance is more than a footnote
Every INDRA factual claim must point to a document ID that opens the archived original. The system also marks sentences that rely on the model's domain knowledge with [DK], go beyond literal reading with [INT], or adopt a requested viewpoint with [PS]. An OCR warning appears when the extracted text may be unreliable.
An ordinary citation does not expose every source boundary. A model may cite a genuine memo while adding a conclusion remembered from training. It may quote real words but attach them to the wrong record. INDRA's quotation watchdog does not ask the model to grade itself. It holds the response, compares quoted text against the loaded OCR with a four-tier fuzzy matcher, and removes unverified material before release.
This resembles the distinction in Content Credentials can verify an image's history, but cannot tell you it is true. Provenance narrows the claim you can make. It does not settle interpretation.
The archive can still lie
The INDRA paper is unusually direct about what its safeguards cannot solve. Its corpus contains documents written by companies anticipating litigation, regulation, or public scrutiny. A memo may be authentic, accurately quoted, and deliberately misleading.
The authors call this the gullibility problem. A model told to stay inside company records may accept a staged expression of concern as sincere. In one test involving Exxon employee magazines, INDRA took the material too literally and described climate as a seriously debated physical phenomenon inside the company. Decades of scholarship provide a wider context that the selected pages did not.
Source selection creates another limit. A researcher who begins with one search phrase chooses the evidence the model can see. Later prompts inherit the vocabulary and assumptions established earlier in the conversation. The paper calls these the Heraclitus effect and the steppingstone dilemma: repeated runs vary, while early framing shapes what later turns can reach.
These limits define the work that remains human. A historian must ask why a record exists, who expected to read it, what is absent, and whether the archive itself was curated under pressure. Resetting a session, changing the query, comparing corpora, and opening the original scan are research methods, not cleanup chores.
A stronger pattern for document AI
An evidence-heavy assistant can adopt the same controls:
- Let the user choose the source set and display exactly what entered it.
- Separate direct quotation, circumstantial pattern, and interpretation.
- Verify quotes and identifiers with deterministic code before generation is released.
- Show OCR quality and keep a link to the original scan.
- Report documents that did not support the claim, not only the matches.
- Cap the session before recall degrades, then make restarting cheap.
That pattern also complements Source Reading Methodology tests the quote and the citation you deleted. In both cases, a fluent summary is secondary. The useful artifact is a chain that another person can inspect.
INDRA is freely available, but the paper's appendices matter as much as the interface because they document the integrity tests and case-study queues. The authors disclose that the platform uses Anthropic's Claude API, received no funding or technical support from Anthropic, and was funded by research and environmental organizations listed in the paper.
Researchers using ish.chat should treat a sourced answer as the beginning of review. Follow the document ID, inspect the scan, rerun the question with a different source set, and look for records that resist the first explanation. INDRA's most useful idea is to label what came from the selected documents and what the model inferred or remembered.
