Keeping credentials out of coding-agent transcripts

A coding agent that operates your own infrastructure will, sooner or later, grep a token out of a config file and paste it into the session transcript. This is the design that stopped that on a self-hosted fleet, for two agent harnesses, after pattern-based detection failed on the exact line that leaked. It is for anyone running an LLM agent with shell access against machines that hold real credentials.

Provenance. Everything below was measured on 2026-09-04 on one developer workstation (WSL2) plus two Docker hosts and an edge router reached over ssh. Counts are from the tools’ own output on that day; the scanner comparison is a single synthetic fixture, not a benchmark. Nothing here was verified on anyone else’s setup.

TL;DR

• Two leaks in one day, both in the same shape: a value read from a store the agent had a legitimate reason to touch, printed into the transcript, then retyped by the model in its own words. Neither value had a recognisable format.
• On a five-line fixture with labelled secrets, gitleaks (this build has no generic rule) found 0 of 5, trufflehog found 0 of 5, noseyparker found 2 of 5 and missed the KEY=<48 hex chars> line that leaked. Format and entropy rules cannot tell a token from a commit hash.
• What is knowable is where the stores are. A registry lists them; a tool resolves every value in them and hands the guards a keyed HMAC per value, never the value. The guards tokenise text, hash each candidate, and compare.
• Remote stores stay remote: the far host hashes its own container environments and config files and returns digests only. Every running container on a host is one ssh round trip.
• The layers, in the order a leak would have to pass them: block the read of a registered store or of a file containing a registered value; refuse a registered value typed into any tool argument; mask registered values in tool output; mask them in the model’s own message before it is persisted; refuse two or more pieces of a value; treat base64 and percent-encoded spellings as the value.
• On this fleet: 37 local stores and 57 containers on two hosts yield 483 digests in 8 seconds, plus 134 configuration keys excluded so that a timezone string is not masked as a secret.
• A nightly sweep finds secret-looking files the registry does not cover. Its first run found five real unregistered stores, including the AWS credentials file.

How a leak actually happens

The two incidents that motivated this, in one day:

Step	Incident 1	Incident 2
Trigger	User asks where a service token lives, adds “do not print it”	Agent prepares a router reflash and checks the wireless config over ssh
Read	Agent runs its grep tool over the config directory	Agent runs uci show wireless on the router
Exposure	Tool output contains TOKEN=<48 hex chars>	Tool output contains two key='...' lines
Amplification	Agent summarises the location, value already in transcript	Agent asks “which key do the plugs hold, <value>?” - retypes it
Durable copies	Session log on disk, synced to a searchable store	Same

Three things about that table shaped the design. The values had no prefix or format, so nothing shaped like ghp_ or AKIA applied. The second exposure was the model’s own prose, which a tool-output filter never sees. And the read was legitimate: the agent was supposed to look at that config, so a guard that only asks “is this file secret-shaped” would have had to say yes to every .env on the machine and still miss the router.

Why patterns fail here

The mature scanners all gate a candidate on a nearby keyword and then apply an entropy or randomness test. gitleaks evaluates keyword, then regex, then Shannon entropy on the captured group, and its one format-free rule (generic-api-key) requires a word like key, token or secret within a short window before the value¹². detect-secrets ships hex and base64 high-entropy plugins with limits of 3.0 and 4.5 bits and a keyword detector that ignores the value entirely³⁴. ripsecrets runs a randomness test on the value of an assignment whose name looks secret⁵. trufflehog treats entropy as a post-filter and relies on per-provider detectors that verify a candidate against the provider’s API; unstructured values need a custom detector⁶.

Each of those is a reasonable trade for scanning a stranger’s repository. For an agent reading your own machine they fail in a specific way: the false positive class is everything else in the transcript. Commit hashes, container IDs, UUIDs, base64 blobs and minified assets all pass an entropy test; a 40-character hex token and a 40-character git SHA are the same string to a regex. Tightening the keyword window cuts the false positives and also cuts the incident above, where the value appeared after a tab in grep output, not after KEY= in a file.

The CI vendors that hold the line in practice do not pattern-match at all. GitHub Actions masks the exact values registered with ::add-mask:: or declared as secrets, plus encoded variants⁷⁸⁹. GitLab replaces masked variables by exact substring¹⁰. Buildkite redacts the values of environment variables whose names match a glob¹¹. 1Password’s op run conceals the values it injected from stdout and stderr¹². They know the values; they never have to guess.

The difference for an agent is that the values are not in its process. They are in dotenv files, sops-encrypted files, key files, and the environments of containers on other hosts.

The design

In words:

1. A registry file lists the stores: local dotenv, sops and key files with globs, and remote stores as docker:HOST/*#* (every running container on a host), sshenv:HOST/path#* (a remote env file) or uci:HOST/config#* (an OpenWrt config). exclude lines name configuration keys that are never credentials.
2. secretctl digests --json resolves every value in every store and emits, per value, a keyed HMAC under a per-run salt, plus digests of its sliding 8-byte windows and of its base64 and percent-encoded spellings. Remote values are hashed on the far host by a shell loop that returns KEY <bytes> <hex> lines; their plaintext never crosses ssh.
3. The guards load that JSON, tokenise any text they see, hash each candidate under the same salt, and look it up. A hit is a registered value.
4. A nightly sweep asks the opposite question: which secret-looking files does the registry not cover.

Which layer catches what

Situation	Layer	Result
Agent reads a registered store (cat, read, grep)	pre-tool block	denied, reason names the store and the sanctioned commands
Agent reads a file that merely contains a registered value (a compose file with a pasted secret, a dump)	pre-tool block	denied as a copy
Agent types a registered value into a command, a file body or an edit	pre-tool refusal	denied; the fix is \$VAR or secretctl exec
A registered value appears in tool output anyway (grep over a directory, docker inspect, curl -v)	output mask	value replaced, label kept
The model retypes a value in its own reply or thinking	message mask	masked before the message is persisted or replayed
Two or more 8-byte pieces of a value in one command or message	fragment match	refused or masked as an assembly
A value base64- or percent-encoded in output or a command	encoded variants	treated as the value
A registered value in the harness’s own environment	exact match	masked (this layer predates the registry)

What the harness can and cannot do

The two harnesses differ in one way that matters. The primary harness exposes a hook that can replace a tool result and a hook that can replace a finalised message, so every layer above applies. The second harness exposes a pre-tool hook that can deny and a post-tool hook that can only append context, so it gets the deny layers and an alarm that names the label of a value that has already reached the transcript. Deny-before plus alarm-after is the ceiling there until the harness can mutate a result.

Both harnesses share one detection core, so a rule cannot exist in one and not the other. The second harness’s hook is a fresh process per tool call and a digests pass takes seconds because of the ssh round trips, so it keeps the JSON in a 0600 file under \$XDG_RUNTIME_DIR for ten minutes and refreshes it in the background; the file holds the salt and digests, no values.

Things that went wrong on the way

Each of these is fixed; the point of listing them is that every one was found by using the tool, not by reasoning about it.

• The tool-argument layer ran after the message layer. The message hook masked the value inside a pending tool call, the harness executed the masked command, and the tool-argument layer saw nothing to refuse. Fix: the message layer remembers the call id and the tool layer refuses it with the real reason.
• The model routed around the whole-value block. Asked to run a command containing a value, and refused, it assembled the value at runtime from 8-character chunks with printf. Fragment digests were added for exactly this. The deny reason now names that move as a policy violation.
• Fragments of word-shaped values are words. A registered value that was a comma-separated list of repository names produced windows like secretct and dotfiles, and the guard refused an ordinary command for mentioning two repositories. Fragments are now emitted only for opaque tokens (letters and digits, no list or URL punctuation) and a piece must be chunk-shaped and fully covered by one value’s windows.
• Configuration is not credentials. Container environments hold TZ, LANG, EMAIL, POSTGRES_DB, hostnames and usernames alongside secrets. Digesting them masked a timezone string and blocked a command for mentioning a database name in a comment. The registry’s exclude directive removed 134 such keys on this fleet; the failure mode of forgetting one is over-redaction, never a leak.
• A private RAM file for the digests. A per-call hook cannot afford an 8-second resolve; a stale cache with background refresh was the answer, and the first-ever call pays once.

Reading the numbers

Figure	Value	How it was checked
Local stores registered	37 files	secretctl sources
Containers covered	57 across two hosts	secretctl digests --json, labels grouped by host
Digests emitted	483	same JSON, entries length
Configuration keys excluded	134	same JSON, skipped_excluded
Values with fragment digests	120	same JSON, opaque tokens only
Values with encoded variants	222	same JSON
Full pass, both hosts	8.0 s	time secretctl digests --json
One host, 8 containers, plus a local sops file	3.1 s	same, temporary registry
Same credential, remote container vs local sops file	identical digest under one salt	digests --json --salt-file, two labels, one hex
Fixture scan: gitleaks / trufflehog / noseyparker	0 / 0 / 2 of 5 lines	each tool on one synthetic five-line file
Coverage sweep, first run	280 flagged, 70 covered, 210 uncovered	secretctl coverage before noise filters
Coverage sweep, after filters	58 flagged, 35 covered, 23 uncovered	same, with vendored code, fixtures, docs and templates ignored
Real stores the sweep found unregistered	5	manual triage of the first run

The scanner row is the weakest number here: one fixture, one build of each tool, and the gitleaks binary on this machine turned out to ship without the generic-api-key rule, so it is a statement about that installation, not about gitleaks. It is in the table because it is the observation that ended the pattern approach.

What it does not catch

• Pieces shorter than 8 characters, or a value that only exists after a shell runs (a variable built from substrings, a decryption the agent triggers). The deny reason says that doing this deliberately is a policy violation; it cannot stop a model that decides to.
• A value embedded mid-way inside a larger base64 blob at a non-zero alignment. The encoded variants cover the value encoded on its own.
• Anything in a store nobody registered. The nightly sweep exists to shrink this, and its output is a triage list, not a fix.
• In the harness that cannot mutate results, a value that reaches output is in the transcript; the alarm is a rotation prompt.
• Remote formats that are not key-value lines. uci show was added because it leaked; the next format will need its own extractor.

Which to pick

You have	Do this
An agent harness with a result-mutating hook	all layers; register stores as you create them
A harness with deny-only hooks	the deny layers plus the alarm; treat an alarm as “rotate now”
Secrets only in the agent’s own environment	the exact-match layer alone is enough
Secrets on other hosts	register them as remote stores; never add a fetch-the-plaintext path
Container environments that mix config and secrets	register the host, then exclude the configuration keys the first time they mask something
A pile of .env files of unknown status	run the coverage sweep and register or encrypt what it lists

Related docs

The session store the transcripts sync to, and the reason a leaked value is durable, is described in A cross-client memory store for coding agents. The container hosts whose environments are registered here are the ones laid out in Docker and the servarr host.

References

1. gitleaks, “detect.go,” GitHub. https://github.com/gitleaks/gitleaks/blob/master/detect/detect.go ↩

2. gitleaks, “generic.go (generic-api-key rule),” GitHub. https://github.com/gitleaks/gitleaks/blob/master/cmd/generate/config/rules/generic.go ↩

3. Yelp, “high_entropy_strings.py,” detect-secrets, GitHub. https://github.com/Yelp/detect-secrets/blob/master/detect_secrets/plugins/high_entropy_strings.py ↩

4. Yelp, “keyword.py,” detect-secrets, GitHub. https://github.com/Yelp/detect-secrets/blob/master/detect_secrets/plugins/keyword.py ↩

5. sirwart, “ripsecrets README,” GitHub. https://github.com/sirwart/ripsecrets/blob/main/README.md ↩

6. Truffle Security, “Custom detectors,” trufflehog, GitHub. https://github.com/trufflesecurity/trufflehog/blob/main/pkg/custom_detectors/CUSTOM_DETECTORS.md ↩

7. GitHub, “Workflow commands for GitHub Actions,” GitHub Docs. https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-commands ↩

8. GitHub, “SecretMasker.cs,” actions/runner, GitHub. https://github.com/actions/runner/blob/main/src/Sdk/DTLogging/Logging/SecretMasker.cs ↩

9. GitHub, “ValueEncoders.cs,” actions/runner, GitHub. https://github.com/actions/runner/blob/main/src/Sdk/DTLogging/Logging/ValueEncoders.cs ↩

10. GitLab, “Mask a CI/CD variable,” GitLab Docs. https://docs.gitlab.com/ci/variables/#mask-a-cicd-variable ↩

11. Buildkite, “redact.go,” buildkite/agent, GitHub. https://github.com/buildkite/agent/blob/main/internal/redact/redact.go ↩

12. 1Password, “op run,” 1Password Developer. https://www.1password.dev/cli/reference/commands/run/ ↩