Chris 在这里说得更透彻:https://x.com/llm_wizard/status/2123158042535411978?a=n12knyRsDNSD3ayy4gM&s=20
Chris said it even better here: https://x.com/llm_wizard/status/2123158042535411978?a=n12knyRsDNSD3ayy4gM&s=20
【引用 @llm_wizard】I feel like people who struggle to understand why benchmarking is hard have never had to interview candidates for a technical job before.
I know the interview process is broken - but it's broken because it's damn near impossible to accurately assess someone completely before hiring them and seeing them do the work.
We're at the stage right now, where the best way to determine if a model is good at what you need it to do - you basically have to bring it on as an intern for a few "weeks" (in model time) just to see if it's a technical and cultural fit.
I feel like people who struggle to understand why benchmarking is hard have never had to interview candidates for a technical job before. I know the interview process is broken - but it's broken because it's damn near impossible to accurately assess someone completely before hiring them and seeing them do the work. We're at the stage right now, where the best way to determine if a model is good at what you need it to do - you basically have to bring it on as an intern for a few "weeks" (in model time) just to see if it's a technical and cultural fit.在 X 查看被引用的帖子
来源:Sebastian Raschka · x.com