Voice Arena 发布 Dictation Bench,这是首个针对 AI 听写应用(而非其 API)的开放基准,用 87 位说话人的 761 段录音在 Mac 上测试 10 款应用,由 5 个 LLM 评委判断草稿能否直接发送。结果显示 Wispr Flow 得分 79%,Apple 仅 35%。该基准按七类使用场景组织,并将需修正草稿的频率与字符编辑总量分开统计。
A dictation benchmark should help someone choose a tool, not require them to become a speech researcher first. The useful starting question is what they need the tool to write.
For a coding workflow, exact technical terms matter. For messages and emails, preserving the intended meaning through restarts and corrections matters. A product can produce fluent text and still fail the task if it changes the detail the user needed to communicate.
@voicearena_ai's Dictation Bench is organized around seven use-case categories rather than one generic passage. It also separates the frequency of drafts needing a correction from the total volume of character edits, and shows examples that make those differences understandable.
If I were choosing a tool, I would use that evidence to build a shortlist, then test it on my own speech. Latency, privacy, and workflow fit would still matter, but I would start with a clearer picture of output quality.
That is the practical significance of a thorough evaluation: it makes a real decision less dependent on a polished demo. The research matters because it helps answer a question people already need to resolve.
Drop 4/15: Announcing the Voice Arena Dictation Bench - the first open benchmark of AI dictation apps. Not their APIs: the real apps. We played 761 recordings from 87 speakers into each of 10 apps on a Mac, then 5 LLM judges asked if each draft could be sent as-is. Wispr Flow: 79%. Apple: 35% 🧵在 X 查看被引用的帖子
来源:elvis · x.com