Turing Post 整理了 Lauren Tan 关于让编码智能体可信的七步做法,包括清理智能体学习的代码、把纠正转化为 lint 规则或测试等可执行检查、限制易误用 API 的使用范围。她所在团队用运行时验证等方式做验证,一个月提交 11000 个 PR,回滚率约 0.5%,并指出若编码循环本身产出垃圾,自动化只会更快产出更多垃圾。
整理自 Lauren Tan 的七步实践,给出让编码智能体产出可信代码的可迁移方法。
How to turn slop into gold. Super concrete steps from awesome @poteto on making coding agents trustworthy:
1. Clean up the code your agents learn from.
Your codebase is one of their biggest inputs. Agents copy whatever looks like an established pattern, including workarounds, deprecated code, and yesterday’s questionable decisions. Clean those up, or watch them reproduce.
2. Turn corrections into enforceable checks.
Every time you correct an agent, ask: can this become a lint rule, a type constraint, a test, or a better API? Put the correction somewhere the system can enforce it. A growing AGENTS.md still requires the agent to follow every instruction.
3. Remove opportunities for predictable mistakes.
Her team @spacexai restricts React’s useEffect to framework internals because agents repeatedly misuse it in product code. They also enforce import boundaries between application processes. These constraints eliminate whole categories of mistakes that would otherwise keep coming back in review.
4. Give agents a reliable way to run and inspect the product.
Provide debugging tools, reusable CLIs, and a “feature map” describing the application from the user’s perspective. Now “that thing in the left sidebar” has a shared meaning, and each agent can reuse the tools for checking it.
5. Review evidence of behavior.
Have agents produce screenshots, videos, and state diagrams. These help you check what the product actually does and how closely that matches your intent. A convincing explanation of a fix is easy to generate; seeing the fix work is considerably more useful.
6. Spend serious effort on verification.
Her team uses runtime verification and is experimenting with property-based UI testing and formal methods, including TLA+ model checking. Lauren says most of her token usage goes toward verification. Agents make some previously expensive testing work much more practical.
7. Automate once that loop is reliable.
Her team now has agents exercise new releases, find bugs, open fixes, and merge them automatically. Getting to that level of trust took her about six months of investing in the environment.
The results she reports: 11,000 pull requests shipped by her team in a month, a revert rate around 0.5%, and improvements in startup, typing, and switching chats.
And what about infamous “software factory”?
if your coding loop produces slop, automating it produces more slop, faster, with a larger token bill.
Start by making good code the easiest thing for an agent to produce. Then scale it.
As always, content at @aiDotEngineer is gold 🏆
来源:Turing Post (Ksenia Se) · x.com