跳到正文
Jim Fan· @DrJimFan · X·· 2026-09-04AI 评分22
AI 导读

Jim Fan 回顾 2016 年在 OpenAI Universe 中的 World of Bits 项目:智能体通过屏幕像素和鼠标操作在 United 上订机票,当时用 Pascal Titan X GPU 从零做 RL、依赖手工逐任务奖励函数,最终未能解决。

正文 · 原文

Good old days at OpenAI in 2016: an agent stares at screen pixels, moves a mouse, and books a flight on United. We called it World of Bits, inside OpenAI Universe. 10 yrs later, Astra is reincarnated in the same universe. Even the naming is astronomically correct 😆

Universe was perhaps the most ambitious AI infra project at the time, but we couldn't quite figure out how to solve it. A policy with zero prior knowledge of what a "submit" button does has to rediscover the entire internet visual lingua by trial and error. In retrospect, RL from scratch against hand-drawn, per-task "artisan" reward functions on a bunch of Pascal Titan X GPUs was completely doomed.

To solve computer use agent, the right way turns out to be boiling the ocean first (hillclimb on every general task you can find), and then specialize back down to the screen pixels and keystrokes. Or simply, a "Specialized Generalist".

Lessons learned: one step ahead of everyone, you're a pioneer. Three steps ahead, you're a prophet. Five steps ahead, you're a martyr.

Congrats, GPT-6! That United flight finally gets booked, reliably this time. The 2016 intern in me has a big smile.

来源:Jim Fan · x.com