morgan: sam：本地推理是对稀缺资源的糟糕分配 “很多人想要本地推理，不知怎么就以为问题已经解决了，因为我们有了开源模型，就能在自己的电脑上跑，于是不再需要数据中心里的那些东西 但从全球资源分配的角度看，让每个人都把世界上有限的 DRAM 供应囤在自己电脑里、只用到 5% 的时间，其实相当糟糕 如果你要应对这种供给极度受限的环境，你需要的是云端利用率最大化 不知为何，很多人的直觉恰恰相反”
morgan:
【引用 @morqon】sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
sam: local inference is a terrible allocation of scarce resources “a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment somehow many people's intuition is the opposite of that”在 X 查看被引用的帖子
来源：Andrew Curran · x.com