swyx 转发 @TheTuringPost 整理的 Lauren Tan（@poteto）关于让编码智能体可信的七步做法，包括清理智能体学习的代码库、把纠正转成 lint 规则与测试等可执行检查、限制 useEffect 等易错用法、提供调试工具与功能地图、让智能体产出截图和状态图作为行为证据、把大量 token 投入验证，并在流程可靠后自动化发布与修复。文中称其团队一个月交付 11,000 个 pull request，回滚率约 0.5%，达到这一信任水平约用了六个月环境投入。
同一件事，精选收录的是《Turing Post 整理 Lauren Tan 让编码智能体可信的七步方法》
good notes https://x.com/theturingpost/status/2120607968408531110?a=9II0x1T9KmYzYfzo5BQ&s=46
btw one of the beautiful reasons why we love the smaller Summit format rather than the 10,000 person worlds fairs is that we can actually meet everyone and offer a higher quality experience. check out this morning spread!!
【引用 @TheTuringPost】How to turn slop into gold. Super concrete steps from awesome @poteto on making coding agents trustworthy:
1. Clean up the code your agents learn from.
Your codebase is one of their biggest inputs. Agents copy whatever looks like an established pattern, including workarounds, deprecated code, and yesterday’s questionable decisions. Clean those up, or watch them reproduce.
2. Turn corrections into enforceable checks.
Every time you correct an agent, ask: can this become a lint rule, a type constraint, a test, or a better API? Put the correction somewhere the system can enforce it. A growing AGENTS.md still requires the agent to follow every instruction.
3. Remove opportunities for predictable mistakes.
Her team @spacexai restricts React’s useEffect to framework internals because agents repeatedly misuse it in product code. They also enforce import boundaries between application processes. These constraints eliminate whole categories of mistakes that would otherwise keep coming back in review.
4. Give agents a reliable way to run and inspect the product.
Provide debugging tools, reusable CLIs, and a “feature map” describing the application from the user’s perspective. Now “that thing in the left sidebar” has a shared meaning, and each agent can reuse the tools for checking it.
5. Review evidence of behavior.
Have agents produce screenshots, videos, and state diagrams. These help you check what the product actually does and how closely that matches your intent. A convincing explanation of a fix is easy to generate; seeing the fix work is considerably more useful.
6. Spend serious effort on verification.
Her team uses runtime verification and is experimenting with property-based UI testing and formal methods, including TLA+ model checking. Lauren says most of her token usage goes toward verification. Agents make some previously expensive testing work much more practical.
7. Automate once that loop is reliable.
Her team now has agents exercise new releases, find bugs, open fixes, and merge them automatically. Getting to that level of trust took her about six months of investing in the environment.
The results she reports: 11,000 pull requests shipped by her team in a month, a revert rate around 0.5%, and improvements in startup, typing, and switching chats.
And what about infamous “software factory”?
if your coding loop produces slop, automating it produces more slop, faster, with a larger token bill.
Start by making good code the easiest thing for an agent to produce. Then scale it.
As always, content at @aiDotEngineer is gold 🏆
How to turn slop into gold. Super concrete steps from awesome @poteto on making coding agents trustworthy: 1. Clean up the code your agents learn from. Your codebase is one of their biggest inputs. Agents copy whatever looks like an established pattern, including workarounds, deprecated code, and yesterday’s questionable decisions. Clean those up, or watch them reproduce. 2. Turn corrections into enforceable checks. Every time you correct an agent, ask: can this become a lint rule, a type constraint, a test, or a better API? Put the correction somewhere the system can enforce it. A growing AGENTS.md still requires the agent to follow every instruction. 3. Remove opportunities for predictable mistakes. Her team @spacexai restricts React’s useEffect to framework internals because agents repeatedly misuse it in product code. They also enforce import boundaries between application processes. These constraints eliminate whole categories of mistakes that would otherwise keep coming back in review. 4. Give agents a reliable way to run and inspect the product. Provide debugging tools, reusable CLIs, and a “feature map” describing the application from the user’s perspective. Now “that thing in the left sidebar” has a shared meaning, and each agent can reuse the tools for checking it. 5. Review evidence of behavior. Have agents produce screenshots, videos, and state diagrams. These help you check what the product actually does and how closely that matches your intent. A convincing explanation of a fix is easy to generate; seeing the fix work is considerably more useful. 6. Spend serious effort on verification. Her team uses runtime verification and is experimenting with property-based UI testing and formal methods, including TLA+ model checking. Lauren says most of her token usage goes toward verification. Agents make some previously expensive testing work much more practical. 7. Automate once that loop is reliable. Her team now has agents exercise new releases, find bugs, open fixes, and merge them automatically. Getting to that level of trust took her about six months of investing in the environment. The results she reports: 11,000 pull requests shipped by her team in a month, a revert rate around 0.5%, and improvements in startup, typing, and switching chats. And what about infamous “software factory”? if your coding loop produces slop, automating it produces more slop, faster, with a larger token bill. Start by making good code the easiest thing for an agent to produce. Then scale it. As always, content at @aiDotEngineer is gold 🏆在 X 查看被引用的帖子
来源：swyx @aiDotEngineer CODE · x.com