跳到正文
elvis· @omarsar0 · X·· 2 小时前AI 评分50
AI 导读

Voice Arena 发布 Dictation Bench,这是首个针对 AI 听写应用(而非其 API)的开放基准,用 87 位说话人的 761 段录音在 Mac 上测试 10 款应用,由 5 个 LLM 评委判断草稿能否直接发送。结果显示 Wispr Flow 得分 79%,Apple 仅 35%。该基准按七类使用场景组织,并将需修正草稿的频率与字符编辑总量分开统计。

正文 · 原文

A dictation benchmark should help someone choose a tool, not require them to become a speech researcher first. The useful starting question is what they need the tool to write.

For a coding workflow, exact technical terms matter. For messages and emails, preserving the intended meaning through restarts and corrections matters. A product can produce fluent text and still fail the task if it changes the detail the user needed to communicate.

@voicearena_ai's Dictation Bench is organized around seven use-case categories rather than one generic passage. It also separates the frequency of drafts needing a correction from the total volume of character edits, and shows examples that make those differences understandable.

If I were choosing a tool, I would use that evidence to build a shortlist, then test it on my own speech. Latency, privacy, and workflow fit would still matter, but I would start with a clearer picture of output quality.

That is the practical significance of a thorough evaluation: it makes a real decision less dependent on a polished demo. The research matters because it helps answer a question people already need to resolve.

引用Shobhit Banga@shobhitbanga
Drop 4/15: Announcing the Voice Arena Dictation Bench - the first open benchmark of AI dictation apps. Not their APIs: the real apps. We played 761 recordings from 87 speakers into each of 10 apps on a Mac, then 5 LLM judges asked if each draft could be sent as-is. Wispr Flow: 79%. Apple: 35% 🧵
在 X 查看被引用的帖子

来源:elvis · x.com