Other professions worked out what a signature has to mean. We're about to need their answer.
A few months ago I had a coding agent build password reset for a small application. It took about five minutes. The email arrived, the link opened, the password changed, the user logged in with the new one.
I reviewed it. I approved it. Everything worked.
The reset link also worked a second time.
The implementation satisfied every behavior I thought to check while missing the invariant underneath them: once the reset succeeds, the token has to become useless. I wrote about that failure as a story about frameworks, because the lesson I took at the time was that mature abstractions carry knowledge forward. That's true. But there's a second thing in that story I walked past.
I was the human in the loop. My approval is on that code. And my approval established nothing.
That is the question I can't put down, and it is not really a question about software.
How do we establish trustworthy human responsibility for work that humans did not directly produce?
This article grew out of a conversation with Sylwia Laskowska following her article, We're Building AI to Replace Ourselves. What's the Plan? We were talking about junior developers, credentialing, and whether portfolios still demonstrate competence when agents can generate polished projects. Sylwia raised the possibility that software engineering might move toward something resembling architecture, with formal qualifications and supervised training. Neither of us was enthusiastic about excluding capable self-taught developers.
I've come around to part of her comparison, though not the part about credentials. What architecture and structural engineering have that we don't isn't a licensing exam. It's a definition of what a signature means.
Somebody already answered this
When a licensed engineer stamps a set of drawings, that stamp is not merely a gesture of approval. It represents a professional claim about the engineer's relationship to the work. Depending on the jurisdiction, that may require direct supervision and control, detailed knowledge of the design, or substantial independent verification and reworking. Simply reading through someone else's drawings and finding nothing obviously wrong is not necessarily sufficient.
Applying a professional seal without meeting the applicable requirements is commonly called plan stamping. Some jurisdictions treat it as grounds for disciplinary action, including fines, suspension, or loss of a professional license. The reasoning is straightforward: a seal is supposed to represent meaningful professional responsibility, not merely the presence of a qualified person's name on a document.
That is roughly a century of institutional thinking about the exact problem AI has just made urgent everywhere else. The professions that stamp things figured out early that a signature is worthless unless it certifies a specific relationship between the signer and the work, and then they went to the trouble of defining the relationship and punishing people who faked it.
Software has the signature. We have the approval on the pull request, the name in the audit log, the reviewer field in the ticket. What we generally lack is a shared definition of what that approval certifies.
The ceremonial approval
Imagine a team using agents to generate hundreds of changes a week, with a senior engineer assigned to approve them. The company can report that every change received human review. If that engineer lacks the time to examine the evidence, the authority to reject anything, or access to the assumptions behind it, the report is accurate and meaningless at the same time.
I don't have to imagine that one. I ran a one-person version of it and shipped a replayable reset token.
This is the failure mode worth naming now, because it arrives disguised as compliance. Nobody sets out to build a ceremonial approval process. It appears when production speeds up and review doesn't, and it is invisible from the outside because the artifacts of oversight are all present. There's a name in the log. The box is checked.
I've made a version of this argument before about code review as a security boundary: review is a good practice that cannot constrain what code is permitted to do, and treating it as a boundary means the boundary doesn't exist. The same shape applies here. An approval is a good practice that cannot by itself establish accountability, and treating it as accountability means accountability doesn't exist.
Responsibility without corresponding authority is not governance. It's liability assignment.
Four conditions
If a signature is going to mean something, the engineering boards suggest what has to be true. Theirs tends to reduce to direct control plus detailed knowledge, or doing the work over. Ours would need four things, and all four, because any one of them missing is enough to hollow out the other three.
An intelligible account. The reviewer needs to know what was proposed, what was checked, what remains uncertain, and what follows from acceptance. "The tests pass" is not that account. My reset flow passed its tests. The tests encoded the same assumption the implementation did.
Competence. The reviewer has to be able to recognize what's wrong, which means they need to have met this class of problem before. This is where the apprenticeship problem bites, and I'll come back to it.
Time. Not nominal availability. Enough time to actually examine the evidence for the volume of work arriving. Verification throughput is a real constraint, and it does not scale with the generation side.
Authority to refuse. The reviewer must be able to say no and have the no hold. A reviewer who can be overridden on schedule pressure is not a check on anything, and everyone in the organization learns that quickly.
Those four are the piece of this I'd want an organization to be able to answer for. Not "do you have human review," which every organization will say yes to, but "what did your reviewer have, and could they have stopped it."
Who gets to decide
There's a second consequence, and it may matter more for how people actually work.
The next professional divide may not run between people who use AI and people who don't. It may run between people authorized to make consequential decisions and people whose work is generated, evaluated, and approved by systems they don't control.
That divide can appear inside software teams, hospitals, design firms, and accounting practices. A small group defines requirements, approves exceptions, and accepts risk. Everyone else operates inside increasingly automated workflows. Some people gain extraordinary leverage. Others keep their jobs and lose the parts of them that involved judgment.
The useful questions are about distribution. Who can challenge the system's recommendation? Who can require more testing? Who can stop a release? Who answers afterward, and did they have the power to change the outcome beforehand?
There's a tempting answer here that I don't think survives contact with incentives: that the displaced production work reappears as verification work. Independent verification, requirements engineering, assurance, evidence management, and competency assessment are all real activities that may well grow. But if one engineer with agents does what five did, nothing guarantees four verification roles appear. A company can simply employ fewer people, underinvest in review, and find out later that its approval process was ceremonial. The need for assurance has never been the same thing as a willingness to pay for it.
The same caution applies to the shorter workweek. Productivity creates the possibility of more leisure. It does not decide who gets it. That's a choice organizations and societies make, not a feature shipped in a model update.
Where the next reviewers come from
Which brings me back to competence, and to Sylwia's original question about junior developers.
Judgment comes from experience. People learn by implementing imperfect solutions, investigating their own failures, absorbing criticism, and living with decisions they made a year earlier. I know a replayable token is a problem because I have spent time on the consequences of that category of mistake.
If agents do the introductory work and companies stop hiring beginners, we get a profession that demands experienced reviewers while closing the path that produces them. Every one of the four conditions above assumes a supply of people competent to review, and we are in the process of disrupting one of the principal mechanisms that has ever generated them.
Credentials don't fix this by themselves. A certificate establishes that someone passed an assessment, not that they'll exercise sound judgment in an unfamiliar situation. And the field has been built substantially by self-taught practitioners, so a licensure regime would exclude people who are currently among its best.
Sustained open-source participation is a better signal than a portfolio, for the reason that the artifact stopped being evidence a while ago. A pull request is another thing an agent can generate. The surrounding history is harder to fake: how someone finds a problem, argues about approaches, responds to review, chases a regression, and supports a change after it lands.
But there's an uncomfortable consequence to leaning on that. If open source becomes the informal proving ground for the profession, the cost of training and evaluating newcomers lands on maintainers who are already donating their time. We don't get to solve the apprenticeship problem by quietly outsourcing it to volunteers.
That one I don't have an answer to. It needs somebody to pay for mentorship who currently isn't.
What I'd ask instead
"Which jobs can AI replace?" sorts professions into safe and unsafe, as though the work inside each one were fixed. The more useful questions are narrower.
Which decisions are getting easier to generate and harder to justify? What evidence should accompany an AI-produced recommendation? Who is permitted to reject it? How does anyone acquire the judgment to do so? And what stops approval from becoming a name in a log attached to a process nobody can inspect?
The answers differ by field. A chatbot feature, a bridge, and a treatment recommendation carry different obligations. But all three turn on the same distinction, which is the one the engineering boards drew a century ago and wrote penalties for: producing an answer and being answerable for it are different acts, and only the second one requires that somebody actually understood the work.
We are issuing an enormous number of signatures right now without having decided what ours certify.
I'd rather we decide on purpose.
Thanks to Sylwia Laskowska for the conversation that prompted this article, particularly her questions about junior developers, credentials, and how people enter the profession as AI changes the work.
Sources on plan stamping: Hawaii DCCA, Notice Regarding Plan Stamping (HAR §16-115-10(4), §16-115-2; HRS §464-10); STRUCTURE magazine, Beware the Stamp.
Top comments (11)
@sylwia-lask Thanks for the back and forth on your post that led to this.
Oh, this is actually a really good point! Maybe this is the missing piece we've been looking for when it comes to programming with AI agents. Of course, we're all basically trying to predict the future here, so take this with a grain of salt. :D
But think about airline pilots. Modern aircraft spend most of their flight time on autopilot. The plane can practically fly itself, and there are countless safety systems in place. And yet, I suspect most people would still be pretty uncomfortable boarding a fully autonomous passenger plane with no human pilots on board.
And here's the interesting part: pilots spend a lot of time training in simulators, learning how to handle emergencies that they might never encounter during an actual flight. They're not just trained to operate the aircraft under normal conditions. They're trained to understand what's happening and take control when something goes terribly wrong.
What if software engineering eventually moves in a similar direction?
Maybe future developers won't need to spend years writing routine code to become experienced engineers. Maybe they'll learn through simulations, deliberately broken systems, security incidents, debugging exercises, and scenarios where AI agents fail in unexpected ways. Instead of learning primarily by writing code, they'll learn by understanding, questioning, and fixing it.
And perhaps that could even help solve the junior developer problem we've been discussing. If the traditional path from junior to senior disappears because agents handle most of the simpler tasks, we'll need another way to train the people who are supposed to supervise those agents.
I have absolutely no idea whether this is how things will actually develop, but I find the possibility fascinating. Maybe the future of software engineering isn't about teaching people to write code faster, but about teaching them what to do when the code-writing machines get something horribly wrong. :D
I really like the pilot-training analogy, and it reminds me of something the technology industry has already experimented with: Cisco's CCIE practical lab exams are one example. Rather than simply demonstrating that you know networking concepts, you're expected to configure and troubleshoot complex networks under real constraints.
Imagine extending that idea to software engineering. Give someone an unfamiliar application and ask them to make it work. Then introduce failures, perhaps while they're away from the environment, and see how they investigate and recover. Maybe the application still runs but has a subtle authorization problem. Maybe the database restores successfully, but its indexes are missing. Maybe an AI agent produces a convincing fix with passing tests that don't actually exercise the broken behavior.
The interesting assessment wouldn't simply be whether they got everything working again. It would be how they identified the problem, which assumptions they challenged, what evidence they gathered, and whether they could explain why the system was now safe to operate.
And I think your aviation comparison adds another dimension: we could deliberately expose engineers to rare but consequential failures that they might otherwise take years to encounter in production. That wouldn't replace the experience of maintaining real systems or living with engineering decisions over time, but it could make developing that judgment much more intentional.
It also connects to the accountability question in the article. If we're going to ask someone to put their name on agent-generated work, perhaps we should assess their ability to recognize when that work isn't ready for approval, not merely their ability to produce more of it.
We might already have some of the educational models we need. We just haven't applied them to this particular problem yet.
I agree with the need for independent verification but I wonder if the bigger challenge is identifying what's missing from the requirements in the first place. Even separate agents building and testing the code could miss the same security issue if neither was asked to consider it. Would threat modeling or some form of independent requirements review help catch these gaps before implementation?
the reset link working twice is the right example, because the review passed every behavior check and still approved nothing about the invariant. we hit the same thing with agent written auth code: the diff reads fine, the happy path works, and nobody asks what must become false after the action. we now make the reviewer attach one negative test per invariant, like token reuse or request replay, before the approval counts. it turns the signature into evidence instead of a click. would you want the stamp to name the invariants the reviewer checked, or only the scope they had authority over?
The reset link working a second time is the detail that makes the whole piece land for me — every behaviour I checked passed, and the invariant underneath just wasn't on my list. That's what makes the stamp question hard: a signature says a human looked, not that they looked for the right thing. Do you see the stamp as covering the checks the reviewer never thought to run, or is that the gap no signature closes?
I think that's exactly the gap no signature can completely close. A stamp shouldn't mean that every possible failure has been anticipated or that the work is guaranteed to be correct. That's an impossible standard, whether a human or an agent did the implementation.
What I'd want it to establish is that the reviewer had the competence, evidence, time, and authority to perform a meaningful assessment, including questioning whether the checks themselves were sufficient. In my password-reset example, the problem wasn't simply that I missed a test. It was that the implementation, the tests, and my review all shared the same incomplete understanding of what a successful reset required.
That's where I think independent verification becomes particularly important. If the reviewer is working from the same assumptions as the system that generated the code, the signature may tell us very little. We can't eliminate every blind spot, but we should at least be able to explain what the approval actually established and what remains uncertain.
This is an important distinction, especially as AI-driven development accelerates.
One concern I'd add is the growing gap between code generation velocity and verification capacity. AI can generate implementation, tests, and even documentation, but when all three are based on the same flawed assumptions, passing tests can create a false sense of confidence.
I believe accountability in AI-assisted engineering must move beyond pull-request approvals toward evidence-based verification: independent security reviews, threat modeling, invariant validation, traceability of AI-generated changes, and clear ownership of production risks.
The point about junior developers is equally significant. If we automate implementation without preserving opportunities to develop engineering judgment, we're creating a future shortage of people capable of validating increasingly autonomous systems.
AI can accelerate engineering execution, but it cannot dilute the responsibility attached to engineering decisions.
The obsession with wrapping remote APIs into multi-layered agent abstractions is creating massive hidden latency and failure cascades in production.
If you profile real-world workloads, bare-metal local inference (via direct C++ / llama.cpp or vLLM bindings) consistently beats fragile cloud wrappers on both predictability and cost. Tooling should eliminate abstraction penalties, not compound them.
The password-reset token story is the perfect illustration of why "I reviewed it" doesn't scale. We hit the identical gap in Electron release sign-off: a human clicking "ship" on an auto-updater build without a checklist of invariants (rollback still works, code signature present, sandbox enabled on the right renderer) is plan-stamping, not review. What moved our bar was moving the invariants the reviewer would have forgotten into the build pipeline itself — a CI hook that re-uses the same update token twice and asserts the second call fails, a hard failure if the shipped binary isn't actually signed. The human stamp then only has to certify what the automation can't see, not what it demonstrably already checked.
A stamp works because it puts a name and a date on a decision that would otherwise drift. That also explains why the plan itself has to be thin enough to actually read before it is stamped. How do you handle a plan that is mostly sound but has one part you cannot honestly endorse?