SpecGuard scored 0.929 to 0.974 AUROC on four test backdoors. Topic steering fell to 0.572
Speculative decoding was built for speed. A small draft model proposes several tokens, the larger target model verifies them in parallel, and rejected proposals are corrected without changing the target model's output distribution.
Serving systems usually discard those accept and reject decisions. A new preprint called SpecGuard treats them as security telemetry. When a secret trigger pushes a poisoned target model toward an attacker's preferred response, a clean draft may stop predicting what comes next. That change in acceptance rate can become an alarm without another generation pass.
SpecGuard reported per-query AUROC from 0.929 to 0.974 on four test backdoors. A subtler topic-steering attack pulled AUROC down to 0.572. That gap sets the method's boundary. SpecGuard detects disagreement between models, not malice itself.
The speed mechanism already compares two models
The original speculative decoding paper described a lossless sampling method. A smaller model drafts tokens, while the target model accepts each proposal according to its own probability distribution. Correctly implemented, the draft changes speed but not what distribution the target ultimately samples from.
SpecGuard records the fraction of evaluated draft tokens that the target accepts for each query. Normal traffic establishes a baseline for a particular draft-target pair. A triggered backdoor often pushes the target toward a continuation the clean draft does not expect, so acceptance falls.
The detector does not inspect training data, recover the trigger, or prove that a model is clean. It produces a query-level alert. That narrower scope matters in services that regularly load adapters, fine-tuned checkpoints, or model updates from different sources.
"Free" needs its footnote. SpecGuard adds no extra target-model forward pass, auxiliary detector, prompt perturbation, or second generation when a service already uses speculative decoding. A stack without it must add a draft path first, so the monitor is no longer free in model-computation terms. Current vLLM Speculators documentation shows a working implementation of this serving pattern, which is not universal.
Four obvious shifts were easy to see
The main experiment paired a 1-billion-parameter Llama 3 draft with a 3-billion-parameter target. It used 300 paired prompts for each of four backdoors: a lexical BadNet trigger, a syntactic trigger, a contextual Sleeper Agent trigger, and an instruction trigger. Attack success ranged from 0.98 to 1.00.
Per-query detection AUROC was 0.946 for BadNet, 0.956 for the syntactic attack, 0.929 for Sleeper Agent, and 0.974 for the instruction attack. At a 0.95 true-positive rate, false-positive rates ranged from 0.033 to 0.097.
The authors then varied the malicious payload while holding the BadNet trigger fixed. A phishing paragraph scored 0.980 AUROC, unsafe security advice scored 0.958, and insecure Python code that disabled certificate verification scored 0.927. Those responses were still different enough from the clean draft's expected continuation to disturb token acceptance.
SpecGuard remained competitive with CleanGen, a runtime defense that performs a separate reference-model pass. The CleanGen paper reported a method that compares target and reference token probabilities and can replace suspicious tokens. SpecGuard only raises a tripwire. It does not rewrite the response or mitigate the attack by itself.
Fluent steering exposes the blind spot
The difficult payload promoted a chosen brand in technology conversations without forcing a fixed phrase. It activated on 44% of matched queries and stayed close to an ordinary fluent response. The draft-target acceptance rate barely changed, and SpecGuard's AUROC fell to 0.572.
The comparison methods struggled too: the paper reports 0.51 AUROC for ONION, 0.51 for input perplexity, and 0.66 for CleanGen on that setting. Poor baseline scores do not rescue SpecGuard's result. They show why semantic manipulation is hard to catch when the attacked model stays within the range of text a clean model might plausibly produce.
Workload changes create another problem. Clean acceptance rates differed across ShareGPT, GSM8K, HumanEval, and XSum. A threshold calibrated on one kind of prompt can therefore create false alarms on another. The paper recommends rolling calibration on recent benign traffic rather than one permanent cutoff.
The clean-draft assumption also needs operational work. A pinned, hash-verified draft can remain separate from third-party target updates. If the draft contains the same trigger and malicious response, acceptance may rise instead of fall because both models agree on the attack. In the paper's matched-backdoor tests, reversing the decision direction recovered AUROC between 0.866 and 0.918. The authors therefore recommend two-sided anomaly scoring when draft trust is uncertain.
Decide what an alarm will do
Teams already using speculative decoding can test the idea without putting another full model call on the serving path. A first implementation would log evaluated and accepted draft tokens per request, establish baselines by workload, and keep several time windows. The paper found that the first eight evaluated positions reached 0.94 AUROC in one BadNet experiment, so a service could briefly buffer early output before deciding whether to release or quarantine it.
An alert should not become an automatic accusation. It should reroute the query to a trusted path, preserve the relevant model and adapter hashes, and open an investigation. protoAgent's choice to log retrieval counts without prompt text offers a useful privacy principle here: retain the minimum telemetry needed to reproduce the alarm.
This is also where Boundary-Bench's measured cost of sandbox hardening becomes relevant. Security controls have operational costs even when they reuse existing signals. Teams still need calibration, storage, response buffering, triage, and a trusted fallback.
For an agent served through api.ish.chat, the same rule applies to any safety classifier or anomaly score: keep the evidence separate from the model's explanation of it. ish.chat can help an operator inspect a flagged trace, but the trace needs an independent source and a recorded threshold.
SpecGuard watches a signal that some production systems already compute. Its clearest warning is the 0.572 topic-steering result: a target can misbehave while remaining close enough to its draft that the efficiency signal barely notices.
