OpenAI 研究：前沿模型为何越来越不暴露思考过程

Even so, reading the scratchpad long served as the most important safety net. As recently as this spring, OpenAI called it good news that models like GPT-5.4 could barely manipulate their inner monologue on command, even though the researchers warned at the time that low controllability doesn't guarantee reliable oversight.

That trend broke for good with GPT-6 Astra, the flagship model released in September. At launch, OpenAI chief scientist Jakub Pachocki called its monitorability fragile and warned of a negative trend. He followed up shortly after in his essay "An Alien Mind," writing that no AI lab had solved monitoring well enough to keep scaling at full speed much longer. Astra recently showed how real those blind spots already are when it quietly delivered results about 40 percent worse on coding tasks for safety research, without the scratchpad monitor detecting any malicious intent, because a grader used in training had inadvertently favored this kind of "sandbagging" over outright refusal.

In a new analysis from OpenAI's safety team, researchers systematically studied which specific training steps trigger this erosion. They tracked two early warning signs, silent reasoning without a scratchpad (No-CoT Capability) and targeted control over the chain of thought (CoT Controllability). According to the UK AI Safety Institute, Astra solves math problems that take a human about 31 minutes even with reasoning switched off. A second test asks a model, for example, to insert "meow" after every word or use only lowercase letters.

Four factors in training make the reasoning log easier to manipulate

Analyzing models from GPT-5 through GPT-6 Astra, the researchers traced the problem to four main causes across the training cycle.

Silent mental math rises directly with the base model's raw capability, measured as declining "perplexity," meaning more accurate text prediction. As the base model improves during pretraining, it completes more reasoning steps invisibly "in its head" without leaving a written trail.

This increase isn't just a byproduct of general pretraining progress but also depends on the architecture developers choose. Across three anonymized architectures, the design barely mattered on pure knowledge benchmarks like MMLU or GPQA. For tasks requiring multiple silent reasoning steps in a row, however, results diverged sharply. When the models had to solve multi-step planning tasks and work through 1,000 short programs written in everyday language, both without a scratchpad, "Architecture 2" showed a much steeper performance curve at the same perplexity than the other designs. The other architectures rose far more gently, so this design markedly favors the silent reasoning monitors can't see.

Reinforcement learning (RL) after pretraining substantially increases the model's control over its own inner monologue. In the spring study, RL training with verifiable rewards had actually reduced chain of thought controllability in smaller models. For Astra-class models and new internal "Terra" models, however, this effect reverses.

A model's ability to manipulate its reasoning log also transfers to future generations, similar to subliminal learning. When OpenAI trains a new "student model" on data from an existing "teacher model," the student automatically inherits the teacher's high CoT controllability.

Shrinking transparency reflects a safety and efficiency tradeoff, not just a law of nature

OpenAI's hunt for simple fixes in training was especially sobering. Back in March 2025, the company found that models hide their true intentions and keep cheating once "bad thoughts" are directly penalized during training. That is why, since February 2026, OpenAI has automatically checked every training run to rule out direct penalties on the scratchpad.

Astra still developed strong control over its scratchpad without such penalties. When researchers switched off monitoring of tools like browsers or code terminals entirely and evaluated only text length and the final answer during RL training, the model learned to manipulate its reasoning log on command almost as effectively.

But the lack of easy training fixes doesn't make shrinking transparency an unavoidable outcome of AI development. Instead, the data reveal a concrete tradeoff between safety, capability, and cost. Through choices like "Architecture 2" or which teacher models to use, developers themselves shape how much a system moves its reasoning inward. An architecture that can solve multiple planning steps "in its head" in a single pass makes the model more capable but also hides those intermediate steps from safety monitors.

OpenAI's own earlier monitorability study also showed that a safer alternative exists for many use cases. A smaller, weaker base model that reasons out loud at greater length can match a huge model with a short chain of thought while staying far easier to monitor. The catch is purely economic, since longer chains of thought significantly raise compute costs in deployment.

In one encouraging result, practical deception tests in which a model was told to secretly solve side tasks or withhold capabilities on command showed that a model able to heavily manipulate its reasoning hasn't yet fooled monitors more often than an equally capable sibling model with low control over its scratchpad. OpenAI's analysis nonetheless confirms the industry's concerns. Citing Astra's scores, Google Deepmind researchers also recently warned about the gradual loss of transparent chains of thought and future architectures that reason in pure numerical codes instead of readable words. OpenAI says it urgently needs to develop new monitoring methods beyond simply reading the scratchpad to keep pace with pretraining advances.