面壁 MiniCPM-OCR 1.0 发布 GGUF 版本，支持 llama.cpp 部署

This repository hosts the GGUF build of MiniCPM-OCR 1.0 for llama.cpp. For the original weights, the full model card, and benchmarks, see the base repository.

MiniCPM-OCR 1.0 (1.3B) parses documents into text, tables, formulas, charts, and seals, and supports key information extraction, for Chinese and English documents and PDFs.

Files

Every tier needs the vision projector as well — --model takes the LLM GGUF, --mmproj takes the projector:

The projector is 4× only; there is no 16× variant for this model.

Which tier

Prefer Q8_0 for code- or XML-heavy pages.

Usage

llama-server \
  --model ./MiniCPM-OCR-1_0-Q8_0.gguf \
  --mmproj ./mmproj-model-f16-4x.gguf \
  --host 0.0.0.0 --port 18499 \
  --ctx-size 40960 \
  --n-gpu-layers 999

--n-gpu-layers 999 offloads everything to GPU; use 0 for CPU-only.

llama-mtmd-cli \
  -m ./MiniCPM-OCR-1_0-Q8_0.gguf \
  --mmproj ./mmproj-model-f16-4x.gguf \
  --image page.png -p "Text Recognition:" \
  --jinja -ngl 99 -c 8192 --temp 0

Thinking is already disabled inside the embedded chat template — no extra flag is needed, and enable_thinking cannot turn it back on. (If you convert the weights yourself instead of using these files, start the server with --chat-template-kwargs '{"enable_thinking": false}'.)

Prompts

SDK with a llama.cpp backend

minicpmvocr parse --config sdk/minicpmvocr/configs/minicpmvocr.yaml \
  --backend llamacpp --server-url http://127.0.0.1:18499 \
  --model minicpm-ocr \
  --set layout.device=cuda:0 \
  --set pipeline.queue_mode=serial \
  --set pipeline.max_workers=1 \
  --input doc.pdf --output doc.md

llama-server handles one request at a time by default, so keep queue_mode=serial and max_workers=1.

Performance

Single RTX 5090 (32 GB), F16 weights: model + projector load ≈2.3 s; one table crop (1,312 prompt / 265 output tokens) ≈1.0 s; VRAM ≈26 GB at --ctx-size 40960. CPU-only (48 threads): one table image end-to-end (layout + recognition + assembly) ≈21 s.

Build llama.cpp and further options: llama.cpp deployment guide.

Related checkpoints

Speculative decoding works in llama.cpp with the DSpark draft GGUF via --spec-type draft-dspark.

FAQ

Installation, model downloads, GPU memory, input formats, and unexpected outputs: FAQ.

License

Model weights, code, and documentation are released under the Apache License 2.0.