xberg
Polyglot document intelligence with a Rust core: extract text, metadata, images, tables, and structured data from 106 formats across 140 file extensions, plus code intelligence for 371 languages. Fifteen bindings, with CLI, REST API, and MCP server.
- 评测生成时间(北京时间)
- 本报告引擎
- v3.10.0
- 当前引擎
- v3.17.3
本报告与当前引擎使用不同规则;原分数不会自动更新,不同版本的分数不宜直接对比。
进入后确认来源与额度,提交才会创建任务。
代码版本未记录或记录无效,无法核对本报告的 commit 与送评文本;不会用当前项目代码回填历史记录。
综合采用结论
证据充分,整体质量与安全表现优秀
- 基础评测完成+25/25确定性评分与静态安全扫描已完成
- README 有效证据+25/2522,905 个去重后的有效字符
- 独立证据来源+20/205 类非重复证据,重复文件不叠加
- 仓库元数据+10/10已取得仓库状态与采用数据
- 活跃记录+5/5已取得最近提交时间
- AI 复核+15/15已完成结构化 AI 证据复核
本报告评了什么
综合分用于静态采用筛选,不是任务成功率,也不代表使用 Skill 后的能力提升。
- 静态检查范围
- 记录 25 个文件 · 142,454 字符
- 只检查采集到的受限文本与已知风险模式;未读取的脚本、依赖和运行环境不在结论内。
- AI 内容复核
- 已使用
- 复核送评文本的说明和设计,不是安装测试或实际任务执行。记录缺失时不推断已使用或未使用。
- 真实任务效果
- 本报告未提供实测证据
- 没有使用/不使用 Skill 的任务对照、重复运行或耗时与 token 成本结果。采用前仍需在隔离环境验证自己的任务。
脚本引用覆盖
脚本引用覆盖记录不可得;不推断没有引用或已经检查。
Xberg 文档智能引擎架构
README展示核心引擎、多种部署模式(库、CLI、REST、MCP)及外部依赖(OCR、嵌入等),无明确时间顺序,适合架构图。
左右滑动查看完整图示
- • 部署模式表列出库、CLI、REST、MCP、Docker
- • OCR后端部分列出Tesseract、PaddleOCR等
- • 嵌入部分说明本地和提供商托管
- README中列出107格式、141扩展名、371语言
- 安装部分提供pip、npm、cargo等命令
- Quick Start展示Rust提取示例
- MCP Server部分列出9个工具、3个提示词、4个资源
- CLI Reference列出14个命令
- 问题与用途描述
- 有效 README
- 安装或接入步骤
- 可执行示例
- 未发现已知高风险模式
- MCP工具的具体输入输出参数未详细说明
- 错误处理和排障信息不足
- 设计细节如权限边界、安全默认值未展示
- 部分功能需额外feature,但未说明默认包含哪些
需要多格式文档提取的开发者、构建RAG管道的团队、需要OCR和代码智能的场景、希望集成到AI助手(如Claude)的用户
需要轻量级单一格式提取的场景、对MCP工具参数有严格要求的集成、需要详细错误处理文档的运维团队
也有自己的公开项目?先免登录查已有报告;没有报告时,可保留地址继续登录评测。
查找我项目的报告 →静态扫描不是安全保证,生产接入前仍应人工复核权限和数据边界。
方法、证据与局限展开收起
GitHub Repository API
25 个文件 · 142,454 字符
v3.10.0 · AI 内容复核:已使用(deepseek-chat)
- 静态评测不会安装或执行项目代码
- 安全扫描基于高信号文件与已知模式,不能替代人工审计
- 流行度只反映采用程度,不代表安全或工程质量
30 天热度趋势
README
Xberg
The fast, precise document-intelligence engine — for every language.
Point Xberg at anything — a PDF, a scanned image, a spreadsheet, an audio file, a URL, a whole archive, or a source tree — and get back clean text, tables, metadata, and structured data. One engine handles format detection, reading, OCR, and extraction, so you never stitch a pipeline together from a dozen libraries.
107 formats · 141 file extensions · 371 code languages · 15 language bindings · 6 output formats · OCR · transcription · embeddings
The fastest, most precise open-source document and PDF-to-Markdown engine — see the benchmarks.
Install · What you get · Capabilities · CLI · Docs
Xberg is the next iteration of Kreuzberg. Same document-intelligence engine, rebuilt and rebranded under a fresh v1 line.
What you get
Point Xberg at anything — a PDF, a spreadsheet, a scanned image, an audio file, a URL, an archive, a source tree — and get back clean, structured content you can use right away. One core does the format detection, reading, and extraction, so you don't assemble a pipeline yourself. Call it from Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, Kotlin, or C FFI, and run it as a library, CLI tool, REST API, or MCP server.
| Capability | What you get |
|---|---|
| 107 document formats | PDFs, Office, images, HTML, email, e-books, scientific publications, and structured data across 141 file extensions, with intelligent MIME detection and bounded extraction controls. |
| URLs & the web | Point Xberg at an http(s) URL — it fetches and extracts a single document, or crawls and follows links (Auto / Document / Crawl modes via the crawlberg engine). Requires the url-ingestion feature. |
| Audio & video transcription | Speech-to-text from MP3, M4A, WAV, WebM, and MP4 tracks via Whisper ONNX (tiny → large-v3). Requires the transcription feature. |
| Archives, traversed | List and recursively extract nested .zip, .tar, .gz, .7z — documents inside documents — guarded by zip-bomb, compression-ratio, and nesting-depth limits. |
| OCR on demand | Tesseract, PaddleOCR, Candle, or VLM backends — fallback chains, confidence scores, language auto-detection, extensible via plugins. |
| Layout & tables | ML layout models (PP-DocLayout-V3, RT-DETR) and table structure (TATR, SLANet) reconstruct reading order and cell grids for clean Markdown. |
| Code intelligence | Functions, classes, imports, symbols, docstrings from 371 programming languages. Syntax-aware chunking for RAG pipelines. |
| Embeddings & search | Local (ONNX) or provider-hosted embeddings (165 providers via liter-llm), sparse and late-interaction, cross-encoder reranking. |
| Enrichment | NER, keyword extraction (YAKE/RAKE), summarization, translation, redaction, page classification, QR detection, language detection, token reduction (TOON). |
| Structured extraction | Schema-driven JSON straight from any document via local (Ollama, LM Studio, vLLM) or hosted LLMs — no prompt engineering. |
| 6 output formats | Plain text, Markdown, Djot, HTML, JSON tree, or Docling DocTags, plus registered custom renderers. |
| Runs anywhere | Library, CLI (14 commands), REST API (xberg serve), MCP server, Docker, Helm — CPU by default, no GPU required. Content-hash caching, parallel batch, per-file timeouts. |
Capabilities marked requires a feature are Cargo feature flags on the core crate (
url-ingestion,transcription,reranker, layout/ORT). Prebuilt language packages and the Docker image bundle the common set; a from-source build enables only what you select.
Installation
Language Packages
Python
pip install xberg
For archived versions and the PyPI storage fallback:
pip install xberg --extra-index-url https://xberg-io.github.io/python-wheels/simple/
PyPI keeps all releases in the current minor and the latest stable patch of each older minor. See the permanent archive and retention policy.
See Python README for full documentation.
Go
go get github.com/xberg-io/xberg/packages/go@latest
⚠️ The repository root is not a Go module —
go get github.com/xberg-io/xbergwill fail. Always target the/packages/gosubdirectory as shown above.
See Go README for full documentation.
Java
Available on Maven Central as io.xberg:xberg. See Java README for the dependency snippet.
Elixir
Add {:xberg, "~> 1.0"} to your mix.exs dependencies. See Elixir README for full documentation.
Kotlin (Android)
Available on Maven Central as io.xberg:xberg-android. See Kotlin README for the dependency snippet.
Swift
Add via Swift Package Manager. See Swift README for full documentation.
Zig
Add via zig fetch. See Zig README for full documentation.
C/C++ (FFI)
Build from source as part of this workspace. See C (FFI) README for full documentation.
CLI & Deployment
CLI Tool
brew install xberg-io/tap/xberg
Windows users can install the same binary through Scoop:
scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install xberg
14 commands: extract, batch, detect, formats, version, cache, tree-sitter, doctor, serve, mcp, api, embed, chunk, and completions.
See CLI usage guide for detailed documentation.
Docker
docker pull ghcr.io/xberg-io/xberg:latest
Run in API, CLI, or MCP modes. See Docker guide for examples.
REST API Server
xberg serve --host 0.0.0.0 --port 8000
One POST endpoint handles all formats. Returns JSON or Markdown. Stream large files. See API server guide.
MCP Server
xberg mcp --transport stdio
9 tools (extract, extract_batch, detect_mime_type, cache_stats, list_formats, cache_clear, get_version, cache_manifest, cache_warm). 3 prompts (extract_document, extract_with_ocr, semantic_search). 4 resources (formats, models, OCR languages, embedding presets).
Add to Claude Desktop or Cursor:
{
"mcpServers": {
"xberg": { "command": "xberg", "args": ["mcp"] }
}
}
AI Coding Assistants
Install the Xberg plugin from xberg-io/xberg. Ships extraction APIs, OCR backends, configuration, and language conventions.
The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.
Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:
npx skills add xberg-io/xberg --list
npx skills add xberg-io/xberg
Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.
Claude Code
/plugin marketplace add xberg-io/xberg
/plugin install xberg@xberg
Codex CLI
/plugins add https://github.com/xberg-io/xberg
Search for xberg and select Install Plugin.
Cursor
Settings → Plugins → Add from URL → https://github.com/xberg-io/xberg, then select xberg.
Gemini CLI
From a checkout of this repository:
gemini extensions install ./plugin
The manifest is inside plugin/. See the Gemini extension reference
for local installation and updates.
Factory Droid
droid plugin marketplace add https://github.com/xberg-io/xberg
droid plugin install xberg@xberg
GitHub Copilot CLI
copilot plugin marketplace add https://github.com/xberg-io/xberg
copilot plugin install xberg@xberg
Pi
Install from a local checkout now:
pi install ./plugin
After the next plugin release, install the same package used by OpenCode:
pi install npm:@xberg-io/opencode-xberg
Pi loads the bundled skills and prompt templates. Use /xberg-extract and /xberg-ocr
to start a task. Install the CLI separately as
shown in the plugin README.
This package does not register MCP servers, hooks, or executable extensions in Pi.
opencode
Add to opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["@xberg-io/opencode-xberg"]
}
Quick Start
Extract text from a document:
use xberg::{extract, ExtractInput, ExtractionConfig};
#[tokio::main]
async fn main() -> xberg::Result<()> {
let config = ExtractionConfig::default();
let output = extract(
ExtractInput::from_uri("document.pdf"),
&config
).await?;
println!("{}", output.results[0].content);
Ok(())
}
Common use cases — see Quick start guide for language-specific examples, OCR, batch processing, and API configuration.
Capabilities
Full feature list
Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)
107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.
Office Documents
| Category | Formats | Capabilities |
|---|---|---|
| Word Processing | .docx, .docm, .doc, .dotx, .dotm, .dot, .odt, .pages, .wpd, .wp, .wp5, .wp6 | Full text, tables, images, metadata, styles |
| Spreadsheets | .xlsx, .xlsm, .xlsb, .xls, .xla, .xlam, .xltm, .xltx, .xlt, .ods, .numbers | Sheet data, formulas, cell metadata, charts |
| Presentations | .pptx, .pptm, .ppt, .pps, .ppsx, .potx, .potm, .pot, .odp, .key | Slides, speaker notes, images, metadata |
.pdf | Text, tables, images, metadata, OCR support | |
| eBooks | .epub, .fb2 | Chapters, metadata, embedded resources |
| Database | .dbf, .sqlite, .sqlite3, .db, .gpkg, .gpkx | Bounded table extraction, schema metadata, GeoPackage detection |
| Hangul | .hwp, .hwpx | Korean document format, text extraction |
Images (OCR-Enabled)
| Category | Formats | Features |
|---|---|---|
| Raster | .png, .jpg, .jpeg, .gif, .webp, .bmp, .tiff, .tif | OCR, table detection, EXIF metadata, dimensions, color space |
| Advanced | .jp2, .jpg2, .j2c, .j2k, .jpc, .jbig2, .jb2, .pnm, .pbm, .pgm, .ppm | OCR via pure-Rust JPEG2000 decoder, JBIG2 support, table detection |
| HEIC family | .heic, .heics, .heif, .heifs, .hif, .avif, .avcs | EXIF metadata, optional pixel decoding |
| Vector | .svg | DOM parsing, embedded text, graphics metadata |
Audio & Video
| Category | Formats | Features |
|---|---|---|
| Audio | .mp3, .mpga, .m4a, .wav, .webm | Whisper transcription |
| MP4 audio track | .mp4, .mpg4, .mp4v, .m4v | Audio-track transcription only |
| MPEG audio track | .mpeg, .mpg, .mpe, .m1v, .m2v | Audio-track transcription only |
| WebM audio track | .webm | Audio-track transcription only |
Web & Data
| Category | Formats | Features |
|---|---|---|
| Markup | .html, .htm, .xhtml, .xht, .xml, .kml, .svg | DOM parsing, metadata (Open Graph, Twitter Card), link extraction |
| Structured Data | .json, .geojson, .jsonl, .ndjson, .yaml, .yml, .toml, .csv, .tsv | Schema detection, nested structures, validation |
| Text & Markdown | .txt, .adoc, .asciidoc, .vtt, .md, .markdown, .commonmark, .qmd, .rmd, .djot, .dj, .mdx, .doctags, .rst, .org, .rtf | AsciiDoc, CommonMark, MyST Markdown, Quarto, R Markdown, Djot, MDX, DocTags, reStructuredText, Org Mode |
Email & Archives
| Category | Formats | Features |
|---|---|---|
.eml, .msg, .pst | Headers, body (HTML/plain), attachments, threading | |
| Archives | .zip, .tar, .tgz, .gz, .7z | File listing, nested archives, metadata, recursive extraction |
Academic & Scientific
| Category | Formats | Features |
|---|---|---|
| Citations | .bib, .ris, .nbib, .enw | Structured parsing: RIS, PubMed/MEDLINE, EndNote XML, BibTeX/BibLaTeX |
| Scientific | .tex, .latex, .typ, .typst, .jats, .nxml | LaTeX, Typst, PubMed JATS |
| Text notebooks | .ipynb, .md, .py, .R, .jl | Jupyter, MyST-NB, Jupytext percent/light, saved outputs, cell visibility tags |
| Publishing | .fb2, .docbook, .dbk, .docbook4, .docbook5, .opml | FictionBook, DocBook XML, OPML outlines |
Code Intelligence (371 Languages)
Extract structure from 371 programming languages via tree-sitter:
| Feature | Description |
|---|---|
| Structure Extraction | Functions, classes, methods, structs, interfaces, enums |
| Import/Export Analysis | Module dependencies, re-exports, wildcard imports |
| Symbol Extraction | Variables, constants, type aliases, properties |
| Docstring Parsing | Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats |
| Syntax-Aware Chunking | Split code by semantic boundaries for RAG pipelines |
| Diagnostics | Parse errors with line/column positions |
Powered by tree-sitter-language-pack.
Output Formats (6)
| Format | Use case | Example |
|---|---|---|
| Plain | Raw text, no markup | "Chapter 1\nIntroduction" |
| Markdown | Readable, structured, RAG-friendly | "# Chapter 1\n## Introduction" |
| Djot | Modern lightweight markup | Similar to Markdown but stricter |
| HTML | Styled, browser-ready | <h1>Chapter 1</h1> |
| JSON | Machine-readable tree structure | Hierarchical sections with heading levels |
| DocTags | Docling-compatible tag stream for document elements and tables | <text>Chapter 1</text> |
Deployment Modes
| Mode | Command | Transport | Use case |
|---|---|---|---|
| Library | xberg::extract() | Async functions | Embed in your application |
| CLI | xberg extract document.pdf | 14 commands | Scripts, batch jobs, CI/CD |
| REST API | xberg serve | HTTP POST | Microservice, serverless deployment |
| MCP Server | xberg mcp | stdio or HTTP | Claude, Cursor, IDE agents |
| Docker | docker run ghcr.io/xberg-io/xberg | All modes | Container deployment |
OCR Backends
- Tesseract — Native C FFI (Linux/macOS/Windows) and WASM (browser)
- PaddleOCR — ONNX Runtime, mobile-optimized models
- Candle — Pure Rust, CPU-only, lightweight
- VLM — GPT-4 Vision, Claude Vision, Gemini Vision, or 165 providers via liter-llm
Fallback chains. Extensible via plugin system.
Embeddings
Local (ONNX Runtime):
- Preset models: fast, balanced (default), quality, multilingual
- Dimensions: 384, 768, 1024
Provider-hosted:
- OpenAI, Anthropic, Google, Hugging Face, Mistral, Cohere, and 165 providers total
- Via liter-llm integration
Reranking:
- Local ONNX rerankers (cross-encoder models)
- Provider-hosted: Cohere Rerank, others
Structured LLM Extraction
Local engines: Ollama, LM Studio, vLLM
Remote: OpenAI, Anthropic, Google, Mistral, Cohere, and 165 providers via liter-llm
Schema validation. Temperature, top-p, frequency penalty tuning.
Enrichment
- NER — GLiNER or LLM-based entity recognition
- Redaction — Mask PII (phone, email, SSN, credit card, addresses)
- Summarization — Document and section summaries via LLM
- Translation — Multi-language via LLM
- Page Classification — Tag document pages (cover, toc, content, etc.)
- QR Code Detection — Extract and decode QR codes from images
- Keyword Extraction — YAKE or RAKE algorithms
- Language Detection — Detect document language
- Layout Detection — RT-DETR + TATR models for document structure
- Table Extraction — Cell-level structure and content
- Token Reduction — TOON wire format (~30–50% fewer tokens than JSON)
CLI Reference
All 14 commands
| Command | Subcommands | Purpose |
|---|---|---|
extract | — | Extract text from a single document (path, URL, or stdin) |
batch | — | Extract from multiple documents in parallel |
detect | — | Identify MIME type of a file |
formats | — | List all supported formats and MIME types |
version | — | Show Xberg version |
cache | stats, clear, manifest, warm | Manage extraction cache and models |
tree-sitter | download, list, cache-dir, clean | Manage code-intelligence grammars |
doctor | — | Diagnose the local installation and runtime dependencies |
serve | — | Start REST API server (default: http://127.0.0.1:8000) |
mcp | — | Start MCP server (stdio or HTTP transport) |
api | schema | Output OpenAPI 3.1 specification |
embed | — | Generate embeddings for text (local or provider-hosted) |
chunk | — | Split text into chunks (text, markdown, YAML, or semantic) |
completions | — | Generate shell completion scripts |
Run xberg --help or xberg <command> --help for detailed options.
Documentation
Full guides, API references for every binding, format reference, and configuration docs live at xberg.io.
- Getting Started
- Quick Start
- Guides
- API Reference (Rust core) — every binding has its own page under
/reference/ - Format Reference
- Live Demo (browser, WASM)
Built with Xberg
Projects that declare Xberg as a dependency. Xberg was previously published as kreuzberg, and most of these projects declare the package under that name.
| Project | What it is | Stars |
|---|---|---|
| basemind | AI context and content layer for coding agents over one MCP server: code map, document RAG, shared memory and web crawl | |
| delulu | A suite of MCP servers and CLI tools that give your LLM better search and fewer hallucinations | |
| docs-mcp-server | Grounded documentation MCP server, an open-source alternative to Context7, Nia and Ref.Tools | |
| erato | The open-source AI platform | |
| fastmail-cli | CLI and MCP server for Fastmail: email, contacts, masked email, attachments and text extraction | |
| ghfdb-portal | Web portal for the Global Heat Flow Database | |
| hawki-toolkit-file-converter | Prepares and converts PDF files for the HAWKI toolkit | |
| haystack-core-integrations | Integrations that extend Haystack with extra components and document stores | |
| kreuzakt | A search engine for humans and computers, aimed at your most boring documents | |
| lilbee | The whole local AI stack in one executable, with conversational search and cited answers over your files, code and the web | |
| llm-workflow-engine | Power CLI and workflow manager for LLMs | |
| MANSPIDER | Spiders entire networks for files sitting on SMB shares, searching filenames or contents with regex | |
| otoroshi-llm-extension | Connect, secure and manage LLM models behind one OpenAI-compatible API | |
| sift-kg | Turns a collection of documents into a knowledge graph, extracting entities and relationships with an LLM | |
| sirchmunk | Turns raw data into a self-evolving, real-time search and intelligence layer | |
| support-chatbot | Level-1 support chatbot for the Netherlands Red Cross 510 team |
Using Xberg in your project? Open a PR adding it to this list.
Contributing
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Join our Discord community for questions and discussion.
Part of Xberg.io
- Xberg — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
- Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
- Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
License
MIT License (MIT) — see LICENSE for details.