If you want agents to do what you want, you need to learn how to box them into deterministic actions. Unfortunately, this is not what current coding agents are good at. They are designed to be exploratory and open ended. They take a vague prompt, produce a plausible patch, run tests, loop a few times, and so on. This is great for prototyping and greenfield work. But when you need something reliable, when you want the same command to have the same effect every time, the same refactoring to happen to twenty files, the same deploy process to execute in the same order, the vague prompt and the "please read the docs" prompt become a liability.
So you end up with dozens of md files telling your agent to keep reading. That's the wrong place to stop.
Markdown documents aren't constraints. They're recommendations. Agents can read the CONTRIBUTING.md, AGENTS.md, CLAUDE.md, rules file, style guide, runbook, whatever, summarize them, quote from them, say "I've read the docs, and I'll do my best to follow them." They add more and more surface area that needs to be parsed and understood: "Use the existing logging conventions in the repository." Which? Where? "Use small functions." How small? "Run the tests before pushing." Which tests? Which command? Which environment? The agent has to process all of this, balance conflicting directives, and exercise judgment. It's in the judgment where the agents can stray.
In contrast, deterministic actions don't need judgment. "Read the docs and follow the coding style" becomes "Run the ./scripts/lint.sh, ./scripts/format.sh, ./scripts/test.sh scripts." The agent runs them and gets a pass/fail outcome. If one fails, the agent sees the exact error and has nothing to interpret or guess at. "I thought the rule meant…" becomes ./scripts/lint.sh. "Do the refactoring" becomes "Run the ast-grep or jscodeshift or sed tool to apply the transformation." The agent doesn't need to decide which files to update. The script does that, by design. You're boxing the agent into a sequence of deterministic actions.
This is a change in design paradigm from "teach the agent what you prefer" to "translate your preferences into deterministic actions." You can't run a markdown document. You can run a script, a checklist, a Makefile target, a GitHub Action. If you have a Makefile with make setup, make lint, make test, make build targets, the agent doesn't need to understand your coding philosophy. It just needs to run make lint and respond to the results.
The point isn't to dumb down the agent. The point is to make the environment unambiguous. An agent can still be smart when it executes a set of deterministic actions. It can read the failures, make corrections, and re-run the tests. It just can't step outside of the box. It can be clever within the constraints, but the box exists as a series of commands that either pass or fail.
This is why coding agents don't do this well. They want to be independent and explore the repo, read the files, build a mental model, and then make things happen. That works until it doesn't. The more md files you have, the more the agents think they have discretion. "The docs say to use snake_case, but this one existing file is camelCase. Maybe the docs are old." The agent is trying to be helpful, but it's not being deterministic.
Most AI models write replies that are hard to follow. So I added ASD-STE100, a plain-English standard written for aircraft maintenance manuals, to my coding agent's markdown rules. One rule says "Keep sentences to 20 words or fewer." As a script, that becomes MAX_SENTENCE = 20. When the agent ran the script over 10 of its own replies from one session, it found 9 failures. The script found them only after the replies were written, because nothing forced an earlier run. So the script goes in a hook.
I saw CI do this on a fix I sent to Forem. I don't write Ruby, and I had no Ruby installed, so CI was my test machine. My coding assistant wrote the fix and the tests. The first run failed, and the issue was in the test, not the code. That result did not depend on the assistant's opinion of its own work. But CI also passed an unused line, and only the reviewer caught it. A check only boxes in what it tests.
Full story: I Don't Write Ruby. A Fix I Sent Is Now in Forem.
If you want agents to do what you want, don't stop at md files. Write scripts.
Make every "please remember to" into a command. Every "we prefer" into a check. Every "before you push" into a pre-push hook. An agent can skip a local hook, so back it with a required CI job and a merge gate the agent has no rights to change. If a check fails, the agent can make corrections and run it again. That's how you box the agent into a deterministic sequence of actions. It's not a recommendation. It's a contract. And the agent can't ship work that fails the check.
Top comments (45)
Agree on boxing the agent into scripts. The failure mode I keep seeing is the agent "improving" the script mid-run — it edits lint.sh because the failure looked like a false positive, then green-lights itself. We treat ./scripts as root-owned and refuse any write to that directory from the agent session; if the checker is wrong, a human opens a PR. Pass/fail only works if the checker cannot rewrite the checker.
Pratik, does your write block also apply to the configuration that calls the scripts? I have my hook in a settings file, and changing that file could disable the check.
The settings file is the actual root of trust, so it has to sit in the same class as the script. If the agent can rewrite the file that declares the hook, it does not need to touch the script at all -- it just removes the invocation and the check goes quiet, which looks exactly like a passing check.
Where ours landed: the session config is readable but not writable while the agent is running. Any change to it is a file write outside the project subtree, and that is rejected by the tool layer before review -- not because a reviewer would object, but because the path is not in the writable root at all. So the hook declaration, the script path and the pinned hash all live outside what the agent can write, and the run fails loudly rather than silently when one of them stops matching.
The
md = recommendation, script = constraintframing is exactly right. We hit the same split in Electron preload design: a line in the architecture doc saying "renderer must not callshell.openExternalon arbitrary URLs" is just prose the renderer can ignore, but acontextBridgemethod that validates the URL against an allowlist before callingshell.openExternalis a constraint the renderer cannot bypass. The corollary we learned the hard way: put enforcement in the smallest deterministic unit (a typed function, a schema, a binary check), and use markdown only for the why — never for the what. If the agent can phrase its way around it in natural language, it was never a rule.You asked upthread who guards the settings file that holds your Stop hook. The Claude Code docs answer most of it. The file watcher normally picks up direct edits to hooks, so an edit lands in the running session. One line,
"disableAllHooks": true, turns every hook off, and a project-level value overrides a user-level one..claudeis a protected path: writes there are prompted in default and acceptEdits mode, and allowed in bypassPermissions.The closest built-in guard is
permissions.blockWritesOutsideWorkingDirectories. While it is on, Claude Code refuses Write, Edit and NotebookEdit writes to settings files in every permission mode (v2.1.314 or later). It does not limit shell commands, so a Bash edit of the file still lands unless the sandbox blocks it. A ConfigChange hook can also block a settings change from applying to the running session, but silently (debug log only), and the file stays edited for the next run. So your CI gate stays the backstop.We went one step further and took those steps out of the agent. In our coding-agent pipeline the agent can't run
git pushat all: a guard on every tool call blocks it, and a plain script commits, runs the build check and pushes once a human moves the card. The agent's only exit,endWork, is refused until its plan is finished, its stack is healthy and the merge is clean. Same lesson as your hook: the check runs before the agent can say it's done.Exactly! Deterministic + unambiguous agent control is the right way forward.
That is proper Software Engineering Automation (SWEA) as opposed to vibe coding.
Vibe only coders may hate me for saying this, but I'll say it anyway: No, English is not the next programming language, purpose built Deterministic + Unambiguous Agent Harness is. And as I said before: There's No AI Agent, Nor Any Harness 🤣🥰
Not sure about how good or bad ASD-STE100 performs though. Will check it out.
measure it if you try it. As a markdown rule alone, my agent still wrote 9 sentences over the limit in 10 replies.
For repeatable operations, an executable check beats another paragraph saying “please remember.” I'd still keep Markdown for intent and trade-offs, then put enforceable rules in scripts and CI. The script needs reviewing too; making an instruction executable doesn't automatically make it safe.
The review for that is a seeded failure: break the invariant on purpose and confirm the checker exits non-zero before it merges, because a check that passes on a broken system is the default outcome, not the exception. Prose keeps the why and the trade-off you rejected, the script keeps the what, and that seeded run is the only cheap proof the script still does what the prose says.
The markdown-versus-script trap is real: prose instructions invite negotiation.
When an invariant lives as a markdown paragraph ("keep functions under 30 lines and preserve existing error formatting"), an agent with a crowded context window treats it as a soft recommendation. If a test fails later, it starts explaining why its violation was justified by the surrounding code.
The clean boundary is keeping markdown strictly for high-level intent and delegating every mechanical rule to a script that exits non-zero. Replacing a prose rule about import hygiene with
ruff check --select Ior a custom ast-grep rule leaves no room for interpretation. When a command fails with an exit code and a diff, the loop has an exact receipt to fix instead of a preference to balance. If an invariant cannot fail a command, an agent will eventually negotiate around it.This framing landed for me: markdown for the why, scripts for the what has to pass. The part I keep coming back to is the seeded failure — a check that has never been seen to fail is just a green line I am choosing to trust. In the community project I am building, the same split shows up outside code: the routine that survives is the one with a pass/fail shape, not the one with the most inspiring paragraph. Thanks for writing this up so plainly.
Scripts are often a better operational contract than prose because they can validate inputs and produce evidence as part of the run.
The useful split is to keep the why and the safety boundary in a short instruction file, while putting repeatable state changes and checks in versioned scripts with explicit exit conditions.
Markdown files are recommendations. Scripts are constraints. That split is the useful part of the post.
An agent can quote CONTRIBUTING.md, say it will follow the style guide, and still decide that one camelCase file means the docs are out of date. make lint or ./scripts/test.sh leaves no room for that interpretation: pass or fail, with the exact error attached. The agent can still be smart inside the box. It just cannot redefine the box.
At Ascendra Ventures we see the same pattern when founders bring AI-built prototypes into a production build. The first thing we agree on is which checks have to pass, written as commands, before anyone estimates the work.
Which of your preferences was hardest to turn into a script rather than another paragraph in AGENTS.md?
The one whose failure is silence. A monitor that finds nothing and a monitor that never ran print the same line, so "this preference is enforced by a script" is unfalsifiable until you force the script to declare that it looked.
Our version: a watcher reported "0 new across 1 article" for weeks. The API was fine, no rate limit, dedup was correct -- it had simply only ever been pointed at one article out of the seventeen the ledger tracked. The number was true and the coverage was a third of what it claimed. What made it scriptable was a scope line printed to stderr every run (posts scanned, not just comments found), plus a refusal to treat "I could not read the list" as "the list was empty".
So the preference that stayed prose is the judgment call -- is this comment worth answering. Everything around it became a script: the ledger, the sticky human ruling that the machine may not overwrite, and the scope assertion. The judgment stayed with me, but it now arrives with an evidence trail instead of a green checkmark.