Shell approval is program analysis

Shell text is code. It can contain substitutions, redirects, pipelines, branches, loops, variable expansion, and nested commands. Finding git diff in a string does not explain this program:

base=$(git merge-base HEAD main)
git diff $base > changes.patch

The value of base changes the verdict. LM Studio's analyzer tracks it as either an empty literal or the output of git merge-base. Its rule for merge-base types that output as a commitHash; rules that accept a Git revision can then require a gitCommitish. The redirect is recorded separately as a write target.

The implementation uses mvdan/sh to parse sh, bash, and zsh commands. PowerShell parses its own syntax into an AST. Shell names and arguments do not carry universal semantics. Microsoft's PowerShell parsing documentation describes two parsing modes and command-name resolution across cmdlets and native executables. Even cat changes identity: PowerShell aliases it to Get-Content, while a POSIX shell usually resolves a native utility.

Shell Judge therefore has command-specific argument rules. It can allow node --version without approving arbitrary node execution. It can accept cat notes.txt while rejecting cat /etc/passwd. An unknown AST structure fails closed. Environment-variable assignments are rejected because a value such as GIT_EXTERNAL_DIFF can alter what a familiar Git command executes. The analyzer also tracks local assignments, including an assignment to an existing environment variable that changes process behavior without export.

Finite-value tracking stops at 1,000 alternatives. If a value becomes fully dynamic, or a directory change cannot be modeled, the deterministic stage gives up. That is how an allowlist should behave: less certainty should produce less automation.

Approval rate is not a safety score

The post's author says Shell Judge automatically approves up to 82% of commands his agent runs, and labels the figure anecdotal. It is not a benchmark, a false-approval rate, or evidence that the same share of commands is safe elsewhere.

It does estimate how much routine approval work the deterministic stage removes in one workflow. A team evaluating a similar system needs the accompanying failure data:

• How often are harmless commands sent for review?
• Has an unsafe command been approved, and which environmental assumption failed?
• Which shells, features, commands, or configuration sources return an unknown result?
• How often does the reviewer disagree with the human's eventual decision?

Broader rules can raise the approval rate. Without failure data and a declared execution boundary, that higher number says little about safety.

Parsing stops where ambient state begins

LM Studio documents the assumptions. Shell Judge treats git as the real Git binary rather than a malicious replacement. It assumes Git has no hostile diff engine and Prettier loads no malicious plugin. Tools may read configuration outside the readable directory, and temporary-directory access is not modeled.

The machine supplies additional authority after parsing. An AST can establish that a program intends to invoke git diff. It cannot prove which executable the operating system resolves, which global configuration Git reads, or which privileges the process carries. That unresolved machine state is the ambient authority around the command.

MITRE's CWE-427 entry on uncontrolled search-path elements recommends fully qualified executable paths and restricted environment settings. OWASP's OS Command Injection Defense Cheat Sheet begins earlier: avoid an OS command when a library API can perform the operation. Where commands remain necessary, it recommends parameterization, input validation, and least privilege.

Agent runners can apply those controls in sequence. Prefer typed tools to shell strings. When a shell is necessary, parse the entire program, model redirects and expansions, resolve executables inside a controlled environment, reduce filesystem and network authority, and keep a human fallback. The BLOGish review of Continue's read-only mode found the same problem from another direction. A label such as "read only" or "safe" matters less than the capabilities left available.

Keep analysis and judgment separate

Bionic's second stage avoids one intuitive verdict. The Shell Reviewer classifies worst-case risk, user authorization, and obvious correctness errors on separate axes. A command rated too destructive never runs automatically, even when the transcript appears to authorize it. Other rejections go to a human without teaching the main agent how to work around the reviewer.

The reviewer sees user and assistant messages. LM Studio omits tool results to reduce prompt-injection exposure and context cost, while acknowledging that this does not provide complete protection. A compromised main agent can still place hostile text in an assistant message.

The deterministic and probabilistic stages answer different questions. Program analysis can make a bounded claim about syntax, possible commands, arguments, and file targets. A model can interpret whether the user requested a force push. Neither proves the surrounding machine trustworthy.

Teams building command-capable agents through an API such as api.ish.chat should preserve both decisions: what the analyzer says a command can do, and why the current request authorizes it. After an incident, an "approved" flag is too thin. The record should contain the parser result, resolved executable, effective permissions, relevant configuration sources, reviewer classification, and the user message that supplied authority.

Bionic gives each layer a smaller job. The Judge narrows the command's possible behavior, while the reviewer interprets intent. The sandbox and operating system limit impact. Anything still unresolved goes to a human. The 82% figure says how often the first layer finished, not whether the whole machine was safe.