Skip to content

Agents and LLMsMethods and reviewreference

How this site is written

Most pages on this site are drafted with coding agents and edited by the author, against a written contract that the build enforces where it can. This page is for anyone running a docs repository the same way. Typography and layout have their own pages, A reading column beside wide tables and One identity, two registers.

Provenance. Counts and file contents were read from the repository on 2026-11-18 at commit fa17865: 196 docs (73 guides, 123 reference), 967 commits since the first on 2025-05-05. Test counts come from one bun test tests run that day. Dates are commit dates. Review findings are quoted from the review notes and commit messages they were recorded in, linked in the evidence table.

TL;DR:

  • One file, the repository’s AGENTS.md (added 2026-07-16), is the authoring contract: doc types, skeletons, citations, house style and the checklist before done. Agent skills send every writer to it first.
  • Two doc types, matching two folders. A guide is a task sequence; a reference doc explains how something works and ends with which option to pick. The sidebar, homepage cards and redirects are generated from each doc’s frontmatter.
  • Citations are numbered GFM footnotes, rendered as a References list. Author-date style was ruled out because a page citing many pages from one vendor turns into a row of identical vendor names.
  • Every structural rule is a test in tests/, and the tests run inside bun run build, so a defect fails the build. At fa17865, before this page was added, the suite was 3,107 tests over 8 files and ran in about 4 s.
  • Wording rules a regex can decide (38 of them) are in the build too, each carrying the sentence it must flag. Sentence shapes a regex cannot decide go to a model in a CI job, and any flag blocks the deploy. Two document-level checks are available as loop sensors and do not run in CI.
  • Pushing to main runs the static and rendered design gates in CI, then the build and test gate, then deploys and purges the cache.
  • Drafts from delegated writers get an adversarial fact-check before publishing. In one six-page series that pass found about 45 wrong or invented claims in drafts that all built cleanly.
writingbefore the commitbun run buildpush to mainAGENTS.md(the contract)agent skillslexicanum, lab-writeup,erfi-voicedraft in aworktreeprose-terms,prose-facts(loop sensors)fact-check pass(delegated drafts)tsc --noEmitbun test testsastro buildbuilt-page testshouse-gate,house-rendered,prose-tellsgit pushbuild, deploy,cache purgeerfi.dev
Kind of ruleWhere it livesWhat breaking it does
Structure a parser can check (tables, footnotes, links, frontmatter, dot style)tests/docs.test.tsfails bun run build locally and in CI
A correction a review proved, or an anchor other docs link totests/pins.test.ts, one row per pinfails the build
A word or phrase shape a regex can decideLLM_MARKERS in tests/lib/mdx.tsfails the build
Term rotation and dropped facts, which need a whole-document tally or a diffloop sensors over .prose-lint.jsonfails the loop iteration; not run in CI
Sentence shapes a regex cannot decidescripts/tell-review.ts in the prose-tells CI job, on changed paragraphs, titles, headings and blurbsblocks the deploy; a wrong flag is cleared in the doc
Layout, contrast and emphasis density on the rendered pagehouse check in two CI jobsblocks the deploy
Whether a claim is truethe fact-check pass and the authornothing automatic

AGENTS.md at the repository root is the authoring system. It has sections for the pipeline facts a writer can trip on, the two doc types, a skeleton for each, the citation convention, house style, conformance for older docs, the checks before done, and the extra rules for pages that publish measured numbers. It names two exemplars, the Cloudflare and Supabase architecture reference and the 10GbE link tuning guide, and tells writers to pattern-match only those. Newer docs are left out as models because a corpus sweep on 2026-08-28 had to remove prose tells from several of them.

Agent skills are on-demand instruction files; the general system is covered in A skills system for coding agents. Three skills and one brief touch this site:

SkillWhat it adds to the contract
lexicanumthe operational layer: commands, traps that have bitten, how to run delegated writers in worktrees, the fact-check pass
writer brief (a file beside the lexicanum skill)the brief every delegated writer gets: read the contract first, take every claim from a file or a read-only command, never commit, and end with a report that cites a source line for each key claim
lab-writeupfor pages built from lab runs: numbers travel from the run artifact by paste, a checklist of ambiguity and attribution classes, and a reviewer brief
erfi-voicethe catalogue of structural AI tells (sentence shapes, independent of vocabulary) that every writer is told to avoid

The skills send writers to AGENTS.md first, but the lexicanum skill also restates several of its rules as traps, and those copies go out of date: on 2026-11-18 it said the inbound-link baseline froze four docs when the file held three. The tests run against the contract and the code, so those take precedence when a skill disagrees.

A guide is a Diataxis how-to: goal-oriented directions for an already-competent reader, in a logical sequence, with anything important but off-task linked rather than explained1. The contract adds that each step should be independently verifiable. A reference doc here is broader than Diataxis reference. It explains how a system works, compares options, carries a TL;DR and measured-vs-asserted labels, and ends with which option to pick. If the question is “how does X work / which do I pick?”, the contract says write a reference doc; if it is “how do I do X, step by step?”, a guide. A guide that rests on a reference doc links it from the lede, and the reference doc links back.

Navigation comes from frontmatter only. Each doc declares a category (one of the providers in src/lib/taxonomy.mjs), a group when that category has subgroups, and optionally featured with a blurb for a homepage card and aliases for old URLs. The sidebar, the homepage card grids and the redirects are generated from those fields when the config loads, and the content schema derives its enums from the same taxonomy, so an unknown category fails the build. A doc cannot invent a category; to add one, edit the taxonomy file. To rename a doc, rename the file and add the old URL to aliases. bun run new prompts for type, category and title and scaffolds the frontmatter and skeleton. The largest categories are split by subject: Supabase and Postgres has eight groups (auth, network access, database, platform services, operations and so on), homelab four, workstations three. Group ids appear in no URL, so regrouping moves no page.

A doc about a system that has since been retired stays published (convention since 2026-10-04). It gets a caution aside under the frontmatter saying what replaced it and which parts still hold, and it loses its homepage card. Other pages link to its URL, and most of what it explains still applies.

A reference doc cites every external factual claim at the claim, as a GFM footnote with a mnemonic slug. The definitions sit in one block near the end of the file, in the form Vendor, "Page title," Site Name. URL. GFM renders footnotes at the bottom of the page wherever they are written2, and a custom rehype pass renames that section to References. Footnotes are numbered by first appearance, which is IEEE style.

The contract rules out author-date style because, on a page that cites many pages from one vendor, every citation becomes the vendor name with a letter suffix, and a reader has to scroll to the list to tell two of them apart. A numbered reference links straight to its entry.

The convention gained three rules later:

  • A definition with no matching inline reference does not render, so every definition must be cited at least once. A test enforces both directions.
  • Links in an evidence table stay inline, even in a reference doc. There the link is the evidence, and a footnote would put the proof one hop away from the claim.
  • A page that cites lab module ids ends with a Modules table naming the experiment, the test file and the published artifact for each id. Module ids collide across experiments (S01 is both security-lockdown and sfp-platforms), and the table is how a reader tells them apart.

Guides keep links inline in the step that uses them and add footnotes only when they make standalone factual claims.

bun run build is four steps: tsc --noEmit, bun test tests, astro build, then the test suite again with CHECK_BUILT=1 for the checks that read the built pages. CI runs the same script before its deploy step, so CI never deploys with a known structural error. The manual path, bun run deploy, uploads whatever dist/ holds and is meant for when CI is broken.

The suite replaced a bash script on 2026-08-03. According to the commit that deleted the script, 29 of its 86 checks duplicated assertions already in the TypeScript suite, over five hardcoded docs instead of the whole corpus, by text matching instead of parsing. Five more could never fail: every cross-link check grepped the built HTML for a slug, and Starlight renders the full sidebar on every page, so every slug matched every page. One of those five asserted that a draft linked to two guides it did not link to at all.

Every check runs over one parse of each MDX file that classifies every line once, as frontmatter, fence, MDX comment, table delimiter or prose, with inline code spans blanked inside prose lines. The bash pattern matches could not tell those apart and produced four wrong checks in one day. The parser has its own tests, and each case is a defect that happened: a dollar sign in frontmatter, one in an indented fence, one in a code span, a footnote reference before a sentence colon. The first run against the real corpus found three bugs in the new checks; once those were fixed, no defects remained in the docs.

The site renders math with remark-math, which treats a single dollar as inline math by default and notes that this often interferes with ordinary dollars in text3, so the dollar check needs every one of those line classes. A price in prose must be written \$10, but the same escape in frontmatter would put a literal backslash into the page description, and inside a fence or code span the dollar is already safe. The check flags only the prose case. One guide renders MTU arithmetic as real math on purpose and opts out with a marker comment, which is also why no test asserts zero math spans site-wide.

GroupChecks
Per docparses with the build’s own MDX parser; no table rendering as pipe text; every table row as wide as its header; no smart punctuation outside code; no unescaped prose dollar; footnote references and definitions balanced and unique; title and description present; a blurb only on featured docs; every dot fence transparent and uncoloured
Across docsevery internal link resolves; no published doc links to a draft; every doc is in the generated sidebar; every doc is linked from another doc or listed in the baseline
Built pagesevery doc produced a page and every page is a doc or a declared redirect; every alias produced a redirect stub; no literal footnote marker leaked; each page’s backlinks list and link graph match the links derived from source; neither is in the search index; only opted-in docs render math
Pinscorrections a review proved, claims that must not come back, required sections, anchors other docs link to
Identifiersno banned identifier anywhere in the corpus
The checks themselvesevery verify:* or check:* script is invoked by the build or a workflow; no test spawns a subprocess

Three of these replaced checks that could not fail. The built-page count used to be a number in AGENTS.md compared by eye; it read “37 docs plus two redirect stubs” when the corpus was 36 docs and three stubs, and summed correctly only by accident. Now a test asserts that built pages minus redirect stubs equal the doc count. The identifier check fails when its list is missing; the bash version reported PASS for weeks on a runner where the list was unset. And no test may spawn a subprocess since a check that shelled out to rg passed locally and died on the CI runner.

The inbound-link check is a ratchet (since 2026-10-21). A new doc must be linked from at least one other doc. Three older docs with no natural host are frozen in tests/no-inbound-baseline.json, and an entry that has since gained a link fails the test too, so the list can only shrink.

Besides required sections and linked anchors, a pin records a correction. A pin that guarded every new fact would break the build on a legitimate rephrase, so a content pin is added when a review proved a claim wrong and the wrong version must not come back.

LLM_MARKERS in tests/lib/mdx.ts is a table of 38 rows on 2026-11-18. Each row is a name, a regex, a sentence the regex must flag, and, where a near-miss exists, a sentence it must not flag. The suite iterates the table, so a rule cannot be added without its cases, and deleting a rule deletes its test with it. The comment above the table explains why: a missing rule and a rule that matches nothing give the same empty output, and the rule against sentence-initial connectives had once been lost without anyone seeing it.

The rows fall into four groups:

  • Vocabulary from the usual AI-prose lists, such as delve, seamless and game-changer, each checked against the corpus for legitimate use first.
  • Importance-announcing frames, such as worth noting and its variants. One row was added after the 2026-08-28 sweep found the banned phrase coming back in other words.
  • The house-style bans on rating your own points (the big win), and on -- in prose, which the Markdown smart-punctuation pass renders as an en dash.
  • Three shapes from review passes on lab-backed pages: an earlier run with no date, a test’s tolerance quoted as if it were the measurement, and is reported to with no source.

Matching runs on prose only. The tenancy reference’s RLS policy carries -- reads as an ordinary SQL comment, and a version of the check that read fences flagged it.

REJECTED_MARKERS, beside the table, lists five phrases that were considered and left out because the corpus already uses them legitimately, two of them in the author’s own writing from before the contract (the key insight among them). They are asserted as non-findings, so adding one later fails next to the reason it was left out. The marker check once had a grandfather list of ten older docs. 139 of their 145 findings were -- in prose, which a code-aware pass fixed mechanically, and six sentences were reworded by hand. The list is empty now, and a test asserts that it stays empty.

Term rotation and dropped facts are available as loop sensors, from prose-lint against .prose-lint.json; an agent loop runs them on the docs its manifest names. They are not in CI.

  • prose-terms fails a doc that says both config and configuration, or database and db. Each word is fine alone, so it takes a tally over the whole document rather than a regex row. Eight synonym sets are configured. After the first run, pairs that were not synonyms in this corpus came out: client means a client library, and in the tenancy docs a customer has users. Removing those cut the findings from 32 to 18 and the false positives to none, and the 18 were frozen in a baseline so only new rotation fails.
  • prose-facts diffs numbers, backticked identifiers and URLs in a doc against HEAD and fails when an edit drops one. Deleting one latency figure from the architecture reference reported “dropped 1 of 270 facts”. It enforces a contract rule: a measured number is never deleted to shorten a sentence, and a number that no longer applies goes in its own commit with the reason.

External link reachability is a test (bun run verify:docs:links) that never gates the build, since another site’s outage is not a defect here. Its first run, over every citation in the corpus, found four dead links; the hand-kept list of six URLs it replaced had included none of them.

Since 2026-11-18 the shapes a regex cannot decide are a CI gate. scripts/tell-review.ts splits each changed doc into running-text paragraphs with the same parser the tests use, skipping tables, fences, imports and footnote definitions, adds the title, each heading and the homepage blurb as their own items, and sends the items a push or pull request touched to a model (gemini-4-argon through OpenRouter, temperature 0) with the instructions in scripts/tell-review-prompt.md. The model returns a kind, an exact quote and a plain rewrite for each flag. A quote that is not in the paragraph is dropped, so an invented flag cannot block a deploy. Any flag that survives fails the job, and build-and-deploy waits for it.

Twelve kinds are defined, from announce-before-stating sentences and aphorisms to self-grading, negative parallelism and metaphors that do not literalise. The instructions also list what not to flag, because the house skeleton itself uses shapes a naive reviewer would catch: the opening sentence saying what a doc is and who it is for, measured-vs-asserted provenance labels, and contrasts that keep two technical things apart. Over all 198 docs, the instructions without those exemptions raised 491 flags and the current ones 280: self-narration fell from 98 to 3, negative parallelism from 60 to 15 and vouching from 29 to 5. In a sample of 30 of the 280, about 25 were tells a careful editor would cut.

A flag the author judges wrong is cleared for that doc with an MDX comment holding the quote, {/* tells-ok: <quote> */}, which does not render. If the model cannot be reached, the job fails (exit 2) rather than passing. The CI key has its own spending cap of 10 USD a month.

The gate was calibrated on this page. Its first draft drew 12 flags under the first version of the instructions and 10 under the current one, all of them tells a hand review against the erfi-voice catalogue had found; the reviewed version drew 2, both real, and none after they were fixed. Only touched paragraphs are reviewed, so the existing corpus blocks nothing until someone edits it. A sweep of all 198 docs on 2026-11-18 found 280 flags in 109 docs. scripts/tell-fix.ts then rewrote the flagged paragraphs with a second model, refusing any rewrite that changed a number, code span, URL, link, footnote or pinned string, kept under 60% of the words, or still drew a flag. Those guards cannot see meaning: a reviewer model reading the word diff sent back 31 of 211 rewrites in the first round and 14 of 77 in the second, for lists that merged, lost qualifiers and dropped provenance labels. A re-sweep afterwards found 90 flags in 53 docs; the sweeps are in docs/research/raw/2026-11-18-tell-sweep/.

erfi-review --changed, a local model on the author’s hardware, was the pass for these shapes before the gate (since 2026-11-17). It prints flags and exits 0, and it still runs as a free check before pushing, but on the same first draft it raised 4 flags, 2 of them real; its probe is in the erfi-review section of the guard extensions page. That page also covers the harness guards that block the regex-able tells, and fold smart punctuation to ASCII, before a file is written at all.

There is no release step. .forgejo/workflows/deploy.yml runs on every push to main and on pull requests, and a docs commit is live within about two minutes of the push.

JobWhat it runsBlocks
house-gatehouse check . --static: design tokens, the vendored house block, the frame audit, the icon buildthe deploy
prose-tellsbun run check:tells --changed <base> on the paragraphs, titles, headings and blurbs the push or PR touched (see below)the deploy
house-renderedbuilds the site in a Playwright image; optical and live checks on two pages, type and layout on every sitemap page (layout at four widths); fails on any skipped step as well as a failed onethe deploy
build-and-deploybun run build with the banned-identifier list from a secret, then wrangler deploy and a cache purge, on main or a manual dispatch-
purge-cloudflare-cachea second, host-scoped purge after a successful deploy-

Pull requests get the gates without the deploy. The Workers assets platform served HTML from a per-PoP cache even with max-age=0, must-revalidate, and without the purge some PoPs served the previous deploy for tens of minutes (observed 2026-08-17). To confirm a deploy, check that the page shows the new content; a 200 status is returned either way.

In house-rendered, one check fails a page when more than 30% of the running text is bold, marked or inline code; on 2026-11-18 a draft with bold list lead-ins and dense code spans reached 32% and the deploy was skipped. A reading column beside wide tables covers the other rendered checks.

Larger pieces are drafted by parallel writer agents, one git worktree each, and the writers never commit. A harness guard blocks git mutations from delegated agents, so the parent reviews each diff and commits it. The review looks for unverified specifics, internal hostnames (replaced with a description unless the corpus already names them), home-directory paths and confidential terms. Cheap writers reliably copy hostnames out of proxy config files and recall version numbers instead of reading them, so both are checked on every diff.

In a six-page series about the agent harness (2026-10-20), every draft built cleanly. Five adversarial fact-checkers, told to assume fabrication and given the measured ground truth, found about 45 wrong or invented claims in them: breakdowns from runs that never happened, incident counts, dates, file counts, code excerpts that did not match the file. In a later series (2026-11-05) the checkers also found errors in the source repository, fixed there in the same pass, and one checker disputed a render the author had screenshotted. The fixer re-verifies every finding before applying it.

Repository-wide review rounds work the same way, with one brief shared by all reviewers. The 2026-10-15 round read 139 docs against four axes, and its repository-level pass found, among other gaps, 29 of 73 guides without a file reference section. The raw notes are kept in the repository under docs/research/raw/.

Two export scripts turn docs into plain Markdown for other hosts. bun run rentry flattens one doc for rentry.co. scripts/pasteriser-export.ts turns a set of docs into a multi-page paste on pasteriser.io: footnotes and dot fences are kept because that host renders both, links between docs in the set become page links, links to other site docs become their text, and escaped dollars are unescaped because that host has no math parser. Anything left that the host would render wrongly is printed to stderr.

Can a parser or a regexdecide it on one lineor one file?Is it a word orphrase shape?yesDoes it need awhole-doc tally ora diff against HEAD?noLLM_MARKERS rowwith hit and missyestest indocs.test.tsno, structurepin, if a reviewproved it wrongno, one doc's claimIs it about therendered page?noloop sensor(prose-lint)yeshouse check(rendered gate)yesprose-tells prompt,or the fact-check passno
  1. A parser or regex can decide it on one line or one file, and it is a word or phrase shape: add an LLM_MARKERS row with a sentence it must flag and, if there is a near-miss, one it must not.
  2. It is decidable and structural (tables, footnotes, links, frontmatter): add a test to tests/docs.test.ts.
  3. It is one doc’s claim that a review proved wrong: add a pin.
  4. It needs a whole-document tally or a diff against HEAD: make it a loop sensor.
  5. It is about the rendered page (layout, contrast, emphasis): it belongs in the house checks.
  6. None of the above: if it is a sentence shape, it goes in the tell-review prompt with an example of what to flag and what not to; if it is about whether a claim is true, it is for the fact-check pass.
ClaimHow it was checked
196 docs, 73 guides, 123 referencefile count under src/content/docs/guides and src/content/docs/reference at fa17865
3,107 tests over 8 files, about 4 sbun test tests at fa17865 on 2026-11-18, before this page: 2,815 pass, 292 skip (the built-page and external-link checks skip without their flags), 0 fail, 4.05 s
38 marker rows, 5 rejected phrasesLLM_MARKERS.length and REJECTED_MARKERS imported from tests/lib/mdx.ts
Bash script: 29 of 86 checks duplicated, five could never failcommit 17d1b95 message, 2026-08-03
Parser cases are real defects; three check bugs found on the first runcommit b541680 message, 2026-08-03
prose-terms 32 -> 18 findings; prose-facts “dropped 1 of 270 facts”commit 821dadc message, 2026-08-05
Grandfather list: 10 docs, 145 findings, 139 of them --comment above MARKER_GRANDFATHERED in tests/docs.test.ts
Three docs in the inbound baselinetests/no-inbound-baseline.json at fa17865
CI jobs, order and conditions.forgejo/workflows/deploy.yml at fa17865
2026-10-15 review: 139 docs, 29 of 73 guides without a file referencedocs/research/raw/2026-10-15-docs-review/meta.md
About 45 wrong claims in six drafts; the 32% emphasis skipthe lexicanum skill’s record of those passes; not published
Contract added 2026-07-16; ratchet 2026-10-21; erfi-review pass 2026-11-17first commit adding each file or rule, from git log
Calibration: 12 then 10 flags on the first draft, 2 then 0 on the reviewed page; erfi-review 4tell-review and erfi-review runs on 2026-11-18 against the pre-edit copy and the committed page
Fixer: 211 and 77 rewrites reviewed, 31 and 14 sent back; 90 flags in 53 docs afterafter-fix-flags.json and the README in docs/research/raw/2026-11-18-tell-sweep/; the review read every word-diff hunk
Corpus sweep: 491 then 280 flags; per-kind counts; about 25 of 30 sampled flags realflags.json and run1-loose-prompt-flags.json in docs/research/raw/2026-11-18-tell-sweep/; the sample was read by hand
  1. Diataxis, “How-to guides,” Diataxis documentation framework. https://diataxis.fr/how-to-guides/ ↩

  2. GitHub, “Basic writing and formatting syntax: Footnotes,” GitHub Docs. https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#footnotes ↩

  3. remarkjs, “remark-math,” README, remark-math repository. https://github.com/remarkjs/remark-math/tree/main/packages/remark-math ↩