跳到正文
elvis· @omarsar0 · X·· 2 天前AI 评分38
AI 导读

MIT 与 NVIDIA 发布 QDA,利用 AI 智能体自动构建并测试候选方案以优化大语言模型量化策略。该方法在 W4A16、W4A8 和 W3A16 精度下分别超越最强公开配方 0.9、1.4 和 2.2 分。QDA 通过测量输出 KL 散度大幅缩短评估时间,并在 GLM-5.3-Flash 上恢复 98.3% 的 FP8 精度。

正文 · 原文

Great paper from MIT and NVIDIA.

I think this is an important step toward personalized, accessible RSI.

The best recipe changes with every model and precision, so experts usually redesign it by hand. QDA hands that loop to an agent that builds and tests candidates over many rounds.

They show that QDA beats the strongest public recipes by 0.9, 1.4 and 2.2 points at W4A16, W4A8 and W3A16, using 64 GPU-hours per search.

Evaluation speed is what makes this work. Scoring one quantized reasoning model normally takes tens of GPU-hours. QDA scores each candidate in minutes by measuring its output KL divergence from the original model on reasoning traces the agent never sees. Promising candidates then get long generations to catch repetition loops.

A shared library of public PTQ methods lets the agent combine them without porting each codebase by hand.

On the 320B MoE GLM-5.3-Flash at NVFP4 W4A4, QDA recovered 98.3% of FP8 accuracy, against 96.9% for the best vendor checkpoint. Every checkpoint runs on existing serving kernels with no inference overhead.

https://arxiv.org/abs/2611.23388

Chat with Paper: https://academy.dair.ai/papers/qda-closed-loop-llm-quantization-design-with-agents-2611.23388

来源:elvis · x.com