protoAgent 0.160.0 logs RAG counts without logging prompt text
Agent observability has an awkward failure mode. The fastest way to explain a bad run is often to record the prompts, retrieved documents, and tool arguments that should never have been copied into a telemetry system.
The small open-source project protoAgent took a narrower route in its 0.160.0 release. It added a caller trust label and RAG pipeline counts to traces and audit records while explicitly excluding credentials, headers, prompt text, chunk contents, configuration, raw task history, and checkpoint data. That does not make the agent safer by itself. It does make two useful questions easier to answer: who initiated the run, and where did retrieved context expand or disappear?
For most agent teams, counts and bounded categories should be the default telemetry. Full content capture should require a separate decision.
Three additions worth separating
The protoAgent 0.160.0 release notes describe three related observability changes.
First, successful authentication already classified callers as operator or federation from the matched credential. The release now copies that classification into request traces and audit logs. The value comes from the authentication result, not a request header or token, and only the two known labels can be recorded. Public requests, preflight traffic, failed authentication, and other unclassified cases retain the previous telemetry shape.
Second, an operator-only GET /api/diagnostics/sessions endpoint returns bounded, redacted summaries of Agent2Agent context. It omits raw task history and checkpoint content.
Third, each RAG injection record can include the effective knowledge top_k, candidate counts after retrieval, after trust filtering, and after final injection, plus the final rag_chunk_ids count. All values come from the same context-projection call. The log stores counts rather than chunk or prompt text.
Trust tier describes the caller. Session inventory describes active context at a safe level of detail. RAG counts show how context moved through a pipeline. Routine diagnosis does not require the underlying conversation.
Why a low-cardinality trust label helps
In a multi-agent system, a local console request and a request from a federated peer may reach the same handler. If their traces look identical, an incident responder has to reconstruct origin from another database or from sensitive request material.
operator and federation are deliberately boring labels. They are bounded, easy to aggregate, and derived after credential matching. OpenTelemetry's guidance for semantic conventions says attributes used for head sampling should generally be low cardinality and available when a span starts. It also says verbose or potentially sensitive attributes should be opt-in. The OpenTelemetry convention guide supports this design, even though protoAgent's field is project-specific.
The label is not authorization. protoAgent's release notes explicitly say the change does not alter authentication, federation denial on /api, Origin checks, or other auth behavior. A dashboard that shows more federation traffic than expected can prompt an investigation. It cannot block that traffic.
Security marketing often blurs observation and enforcement. We saw the same problem in prompt-injected payment runs: detection and disclosure were part of the failure, but better logs alone would not have stopped the action.
Count the RAG stages before storing their contents
RAG failures often appear as a bad answer, but the fault may occur earlier. Retrieval may return too many candidates. A trust filter may remove nearly everything. The final projection may ignore its intended cap. Four counts can show which stage changed the set without copying the documents into a trace backend.
Those numbers support alerts on patterns such as:
- retrieved candidates suddenly increasing while
top_kstays fixed; - the trust filter removing none of the candidates from an untrusted source;
- the injected count exceeding the expected context budget;
- the
rag_chunk_idscount disagreeing with the injected count.
The release is careful about scope. These fields are forensic provenance only. They do not change retrieval order, filtering, the top_k cap, or the composed prompt. They also do not reproduce or fix the earlier reported 26-chunk over-injection incident. Legacy rows decode with explicit defaults.
A new metric can make an old bug visible without proving the bug is gone.
Content logging should be an exception
OpenTelemetry's generative AI attribute registry defines fields for input messages, output messages, retrieval documents, queries, system instructions, tool arguments, and tool results. It also warns that input messages are likely to contain sensitive information, including personal data.
Those fields can help during a controlled evaluation. In production, they can duplicate customer text, internal instructions, retrieved records, and secrets across exporters and observability vendors. Redaction after collection is weaker than never collecting the content.
A practical default records identities by opaque ID, pipeline stage counts, policy decisions, duration, error type, token use, and whether human approval occurred. If a short debugging window needs content capture, make it explicit, sampled, access controlled, and time limited.
The protoAgent repository describes JSONL audit logs with trace IDs and OpenTelemetry tracing. Teams using api.ish.chat, ish.chat, or another agent gateway can apply the same split at the boundary: operational metadata by default, content only under a documented debugging policy.
Logs do not reduce agency
OWASP's Excessive Agency guidance lists malicious or compromised peer agents alongside prompt injection and compromised extensions as paths to unsafe action. Its root causes include excessive functionality, permissions, and autonomy.
A trust-tier field helps investigators separate peer traffic from operator traffic. RAG counts help them locate unexpected context. Neither narrows a tool's permissions or inserts human approval before a consequential action. Those controls still belong in the execution path.
The implementation has separate layers. Derive trust after authentication and attach a small categorical label to the trace. Record how many items survive each context stage. Enforce permissions before the tool runs, then test the control with adversarial inputs and replayed incidents, as in the shadow-run approach covered in TRUSS.
protoAgent 0.160.0 does not solve agent security. Its narrower contribution is enough: telemetry can explain the shape of a run without becoming a second archive of everything the agent saw.
