(2027年的RSI)
(RSI in 2027)
【引用 @scaling01】OpenAI just gave us a glance behind the curtain.
This is how much models have improved in terms of log perplexity or the average cross-entropy loss per token, which measures how well a base model predicts the next token
Jakub Pachocki says: "as soon as 2027 we could conceivably see a similar avalanche of fundamental pretraining advancements" (similar to the advances in mathematics)
He also warns that: "While I expect AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining."
Jakub also calls for pacing with "common limits" for AI labs to make sure we first understand and are able to monitor new, even more capable systems.
OpenAI just gave us a glance behind the curtain. This is how much models have improved in terms of log perplexity or the average cross-entropy loss per token, which measures how well a base model predicts the next token Jakub Pachocki says: "as soon as 2027 we could conceivably see a similar avalanche of fundamental pretraining advancements" (similar to the advances in mathematics) He also warns that: "While I expect AI will also significantly contribute to progress on alignment and monitoring, I think that progress is unlikely to be fast enough to allow the full potential acceleration in pretraining." Jakub also calls for pacing with "common limits" for AI labs to make sure we first understand and are able to monitor new, even more capable systems.在 X 查看被引用的帖子
来源:Lisan al Gaib · x.com