Configuration Guide

All extraction behavior is controlled through ExtractionConfig. Pass it directly in code or load it from a TOML/YAML/JSON file. Every field is optional. For per-field documentation, see the Configuration Reference.

Quick Start

Python

Python

import asyncio
from xberg import ExtractInput, extract, ExtractionConfig

async def main() -> None:
    config = ExtractionConfig(
        use_cache=True,
        enable_quality_processing=True
    )
    result = await extract(ExtractInput(uri="document.pdf"), config)
    print(result.results[0].content)

asyncio.run(main())

TypeScript

TypeScript

import { extract } from "@xberg-io/xberg";

const config = {
  useCache: true,
  enableQualityProcessing: true,
};

const output = await extract({ kind: "uri", uri: "document.pdf" }, config);
console.log(output.results[0].content);

Rust

Rust

use xberg::{extract, ExtractionConfig, ExtractInput};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig {
        use_cache: true,
        enable_quality_processing: true,
        ..Default::default()
    };

    let output = extract(ExtractInput::from_uri("document.pdf"), &config).await?;
    println!("{}", output.results[0].content);
    Ok(())
}

Go

Go

package main

import (
  "log"

  "github.com/xberg-io/xberg/packages/go"
)

func main() {
  useCache := true
  enableQP := true

  cfg := xberg.ExtractionConfig{
    UseCache:                &useCache,
    EnableQualityProcessing: &enableQP,
  }
  input := xberg.ExtractInputFromURI("document.pdf")
  result, err := xberg.Extract(*input, cfg)
  if err != nil {
    log.Fatalf("extract failed: %v", err)
  }

  log.Println("content length:", len(result.Results[0].Content))
}

Java

Java

import io.xberg.Xberg;
import io.xberg.ExtractInputKind;
import io.xberg.ExtractionResult;
import io.xberg.ExtractedDocument;
import io.xberg.ExtractionConfig;
import io.xberg.ExtractInput;

ExtractionConfig config = ExtractionConfig.builder()
    .useCache(true)
    .enableQualityProcessing(true)
    .build();
ExtractionResult output = Xberg.extract(
    ExtractInput.builder().withKind(ExtractInputKind.Uri).withUri("document.pdf").build(),
    config
);
ExtractedDocument result = output.results().get(0);

C#

C#

using Xberg;

var config = new ExtractionConfig
{
    UseCache = true,
    EnableQualityProcessing = true
};

var result = (await XbergConverter.ExtractAsync(ExtractInput.FromUri("document.pdf"), config)).Results[0];
Console.WriteLine(result.Content);

Ruby

Ruby

require 'xberg'

config = Xberg::ExtractionConfig.new(
  use_cache: true,
  enable_quality_processing: true
)

input = Xberg::ExtractInput.new(uri: 'document.pdf')
result = Xberg.extract(input, config)

Configuration Files

Three formats are supported. TOML is recommended.

TOML (Recommended)

xberg.toml

use_cache = true
enable_quality_processing = true

[ocr]
backend = "tesseract"
language = "eng"

[ocr.tesseract_config]
psm = 3

YAML

xberg.yaml

use_cache: true
enable_quality_processing: true

ocr:
  backend: tesseract
  language: eng
  tesseract_config:
    psm: 3

JSON

xberg.json

{
  "use_cache": true,
  "enable_quality_processing": true,
  "ocr": {
    "backend": "tesseract",
    "language": "eng",
    "tesseract_config": {
      "psm": 3
    }
  }
}

Automatic Discovery

When no --config path is supplied, Xberg walks up from the current working directory looking for xberg.toml and uses the first match. If no project-local file is found, it falls back to a per-user global config at xberg/xberg.{toml,yaml,yml,json} in the platform config directory — $XDG_CONFIG_HOME (or ~/.config) on Linux, ~/Library/Application Support on macOS, and %APPDATA% on Windows. In the project walk, YAML and JSON files are supported only when passed explicitly via --config. If nothing is found, defaults are used.

Python

Python

import asyncio
from xberg import ExtractInput, ExtractionConfig, extract

async def main() -> None:
    config: ExtractionConfig = ExtractionConfig()
    result = await extract(ExtractInput(uri="document.pdf"), config)

    content: str = result.results[0].content
    content_preview: str = content[:100]

    print(f"Content preview: {content_preview}")
    print(f"Total length: {len(content)}")

asyncio.run(main())

TypeScript

TypeScript

import { extract, type ExtractionConfig } from "@xberg-io/xberg";

// Note: the Node binding has no config-file discovery helper. Build the
// config object directly (or load `xberg.toml`/`xberg.yaml`/`xberg.json`
// yourself and parse it) and pass it to `extract`.
const config: ExtractionConfig = {
  useCache: true,
};

const output = await extract({ kind: "uri", uri: "document.pdf" }, config);
console.log(output.results[0].content);

Rust

Rust

use xberg::{extract, ExtractionConfig, ExtractInput};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::discover()?.unwrap_or_default();
    let output = extract(ExtractInput::from_uri("document.pdf"), &config).await?;
    println!("{}", output.results[0].content);
    Ok(())
}

Go

Go

package main

import (
  "log"

  "github.com/xberg-io/xberg/packages/go"
)

func main() {
  config, err := xberg.LoadExtractionConfigFromFile("")
  if err != nil {
    log.Fatalf("discover config failed: %v", err)
  }

  input := xberg.ExtractInputFromURI("document.pdf")
  result, err := xberg.Extract(*input, *config)
  if err != nil {
    log.Fatalf("extract failed: %v", err)
  }

  log.Printf("Content length: %d", len(result.Results[0].Content))
}

Java

Java

import io.xberg.Xberg;
import io.xberg.ExtractInputKind;
import io.xberg.ExtractionResult;
import io.xberg.ExtractedDocument;
import io.xberg.ExtractionConfig;
import io.xberg.ExtractInput;

ExtractionConfig config = Xberg.discoverExtractionConfig();
ExtractionResult output = Xberg.extract(
    ExtractInput.builder().withKind(ExtractInputKind.Uri).withUri("document.pdf").build(),
    config
);
ExtractedDocument result = output.results().get(0);

C#

C#

using Xberg;

var config = new ExtractionConfig();
var result = (await XbergConverter.ExtractAsync(ExtractInput.FromUri("document.pdf"), config)).Results[0];

Console.WriteLine(result.Content[..Math.Min(100, result.Content.Length)]);
Console.WriteLine($"Total length: {result.Content.Length}");

Ruby

Ruby

require 'xberg'

config = Xberg::ExtractionConfig.discover
input = Xberg::ExtractInput.new(uri: 'document.pdf')
result = Xberg.extract(input, config)

Wasm

WASM

import { initWasm, extract } from "@xberg-io/xberg-wasm";

await initWasm();

const config = {
  use_cache: true,
  enable_quality_processing: true,
  ocr: {
    backend: "tesseract-wasm",
    language: "eng",
  },
};

const bytes = new Uint8Array(buffer);
const result = await extract({ kind: "bytes", bytes, mimeType: "application/pdf" }, config);
console.log(result.content);

Environment Variable Overrides

ExtractionConfig::apply_env_overrides() applies XBERG_* variables on top of an already-loaded config. Each variable that is set overrides the matching config-file value; unset variables are ignored. The serve and mcp commands call it automatically after loading the config. The extract and batch commands do not apply it — use flags or --config-json there.

Variable	Overrides
XBERG_OCR_LANGUAGE	OCR language (ISO 639 code, e.g. eng, deu)
XBERG_OCR_BACKEND	OCR backend (tesseract, paddle-ocr, sceptre, vlm)
XBERG_DISABLE_OCR	Disable OCR entirely (true/false)
XBERG_CHUNKING_MAX_CHARS	Maximum characters per chunk
XBERG_CHUNKING_MAX_OVERLAP	Overlap between chunks
XBERG_CHUNKING_TOKENIZER	HuggingFace tokenizer model ID for token-based sizing
XBERG_CACHE_ENABLED	Cache flag (true/false)
XBERG_TOKEN_REDUCTION_MODE	Token reduction level (off, light, moderate, aggressive, maximum)
XBERG_OUTPUT_FORMAT	Output format
XBERG_LAYOUT_PRESET	Layout detection preset (fast, accurate)
XBERG_LLM_MODEL	LLM model for structured extraction
XBERG_LLM_API_KEY	API key for the structured-extraction LLM provider
XBERG_LLM_BASE_URL	Custom base URL for the LLM provider
XBERG_VLM_OCR_MODEL	VLM model for vision-based OCR
XBERG_VLM_EMBEDDING_MODEL	LLM model for embedding generation
XBERG_EMBEDDING_PLUGIN_NAME	Name of a registered in-process embedding backend

Server-only variables (XBERG_HOST, XBERG_PORT, XBERG_CORS_ORIGINS, XBERG_MAX_REQUEST_BODY_BYTES, XBERG_MAX_MULTIPART_FIELD_BYTES) configure the API/MCP server, not extraction.

Loading Precedence

For the extract and batch commands, sources are applied highest to lowest:

1. Individual CLI flags (--ocr, --output-format, --chunk, …)
2. Inline JSON (--config-json or --config-json-base64) — merged field by field, not whole-object
3. Config file — explicit --config, otherwise the auto-discovered xberg.toml
4. Built-in defaults

The serve and mcp commands add environment variables on top of the loaded config via apply_env_overrides(), so a set XBERG_* variable overrides the config-file value in those modes.

Common Use Cases

Setting Up OCR

Python

Python

import asyncio
from xberg import ExtractInput, ExtractionConfig, OcrConfig, TesseractConfig, extract

async def main() -> None:
    config: ExtractionConfig = ExtractionConfig(
        ocr=OcrConfig(
            backend="tesseract", language="eng+fra",
            tesseract_config=TesseractConfig(psm=3)
        )
    )
    result = await extract(ExtractInput(uri="document.pdf"), config)
    print(result.results[0].content)

asyncio.run(main())

TypeScript

TypeScript

import { ExtractInputKind, extract } from "@xberg-io/xberg";

const config = {
  ocr: {
    backend: "tesseract",
    language: ["eng", "fra"],
    tesseractConfig: {
      psm: 3,
    },
  },
};

const output = await extract(
  {
    kind: "uri",
    uri: "document.pdf",
  },
  config,
);

console.log(output.results[0].content);

Rust

Rust

use xberg::{ExtractionConfig, OcrConfig, TesseractConfig};

fn main() {
    let config = ExtractionConfig {
        ocr: Some(OcrConfig {
            backend: "tesseract".to_string(),
            language: "eng+fra".to_string(),
            tesseract_config: Some(TesseractConfig {
                psm: 3,
                ..Default::default()
            }),
            ..Default::default()
        }),
        ..Default::default()
    };
}

Go

Go

package main

import "github.com/xberg-io/xberg/packages/go"

func main() {
  psm := int32(3)

  _ = xberg.ExtractionConfig{
    Ocr: &xberg.OcrConfig{
      Backend:  "tesseract",
      Language: "eng+fra",
      TesseractConfig: &xberg.TesseractConfig{
        Psm: &psm,
      },
    },
  }
}

Java

Java

import io.xberg.ExtractionConfig;
import io.xberg.OcrConfig;
import io.xberg.TesseractConfig;

ExtractionConfig config = ExtractionConfig.builder()
    .ocr(OcrConfig.builder()
        .backend("tesseract")
        .language("eng+fra")
        .tesseractConfig(TesseractConfig.builder()
            .psm(3)
            .build())
        .build())
    .build();

C#

C#

using Xberg;

var config = new ExtractionConfig
{
    Ocr = new OcrConfig
    {
        Backend = "tesseract",
        Language = "eng+fra",
        TesseractConfig = new TesseractConfig { Psm = 3 }
    }
};

var result = (await XbergConverter.ExtractAsync(ExtractInput.FromUri("document.pdf"), config)).Results[0];
Console.WriteLine(result.Content);

Ruby

Ruby

require 'xberg'

config = Xberg::ExtractionConfig.new(
  ocr: Xberg::OcrConfig.new(
    backend: 'tesseract',
    language: 'eng+fra',
    tesseract_config: Xberg::TesseractConfig.new(psm: 3)
  )
)

For backend selection and language packs, see OCR Guide. For fine-grained Tesseract tuning, see TesseractConfig Reference.

For Sceptre, use the backend name sceptre and place its sections directly in backend_options:

xberg.toml

[ocr]
backend = "sceptre"
language = ["eng", "deu"]

[ocr.backend_options.recognition]
batch_size = 4

[ocr.backend_options.concurrency]
max_threads = 2

Do not nest these options under backend_options.sceptre. Custom Rust builds use sceptre-ocr for the ONNX Runtime desktop/server backend or sceptre-ocr-tract for Android and iOS. WebAssembly support uses the opt-in Sceptre worker build/API because the model and tract runtime size are not part of the default wasm-target bundle. Use ocr.language for language selection; it overrides backend_options.model.languages. Leave backend_options.model.backend unset so Xberg can select ORT or tract for the target. The supported option paths are backend_options.detection.*, backend_options.recognition.*, backend_options.concurrency.max_threads, backend_options.model.cache_dir, backend_options.model.registry_owner, backend_options.model.detector_path, and backend_options.model.recognizer_path. cache_dir and automatic model download apply only to desktop/server ORT builds. Android and iOS exclude the downloader; both model paths are required and must reference application-resolved bundle or asset files.

Chunking for RAG

Python

Python

from xberg import (
    ExtractionConfig,
    ChunkingConfig,
    EmbeddingConfig,
    EmbeddingModelType,
)

config: ExtractionConfig = ExtractionConfig(
    chunking=ChunkingConfig(
        max_characters=1500,
        overlap=200,
        embedding=EmbeddingConfig(
            model=EmbeddingModelType.preset("balanced")
        ),
    )
)

TypeScript

TypeScript

import { extract } from "@xberg-io/xberg";

const config = {
  chunking: {
    maxCharacters: 1500,
    overlap: 200,
    embedding: {
      model: { type: "preset", name: "quality" },
    },
  },
};

const output = await extract({ kind: "uri", uri: "document.pdf" }, config);
console.log(`Chunks created: ${output.results[0].chunks?.length ?? 0}`);

Rust

Rust

use xberg::{ChunkingConfig, EmbeddingConfig, EmbeddingModelType, ExtractionConfig};

fn main() {
    let config = ExtractionConfig {
        chunking: Some(ChunkingConfig {
            max_characters: 1500,
            overlap: 200,
            embedding: Some(EmbeddingConfig {
                model: EmbeddingModelType::Preset {
                    name: "text-embedding-all-minilm-l6-v2".to_string(),
                },
                ..Default::default()
            }),
            ..Default::default()
        }),
        ..Default::default()
    };
    println!("{:?}", config.chunking);
}

Go

Go

package main

import (
  "fmt"
  "log"

  "github.com/xberg-io/xberg/packages/go"
)

func main() {
  maxChars := 1000
  maxOverlap := 200
  cfg := xberg.ExtractionConfig{
    Chunking: &xberg.ChunkingConfig{
      MaxChars:   &maxChars,
      MaxOverlap: &maxOverlap,
    },
  }

  input := xberg.ExtractInputFromURI("document.pdf")
  result, err := xberg.Extract(*input, cfg)
  if err != nil {
    log.Fatalf("extract failed: %v", err)
  }

  for i, chunk := range result.Results[0].Chunks {
    fmt.Printf("Chunk %d/%d (%d-%d)\n", i+1, chunk.Metadata.TotalChunks, chunk.Metadata.CharStart, chunk.Metadata.CharEnd)
    fmt.Printf("%s...\n", chunk.Content[:min(len(chunk.Content), 100)])
  }
}

func min(a, b int) int {
  if a < b {
    return a
  }
  return b
}

Java

Java

import io.xberg.ChunkingConfig;
import io.xberg.EmbeddingConfig;
import io.xberg.EmbeddingModelType;
import io.xberg.ExtractionConfig;

ExtractionConfig config = ExtractionConfig.builder()
    .chunking(ChunkingConfig.builder()
        .maxChars(1500)
        .maxOverlap(200)
        .embedding(EmbeddingConfig.builder()
            .model(EmbeddingModelType.builder()
                .type("preset")
                .name("text-embedding-all-minilm-l6-v2")
                .build())
            .build())
        .build())
    .build();

C#

C#

using Xberg;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

var config = new ExtractionConfig
{
    Chunking = new ChunkingConfig
    {
        MaxCharacters = 512,
        Overlap = 50,
        Embedding = new EmbeddingConfig
        {
            Model = new EmbeddingModelType.Preset("balanced"),
            Normalize = true,
            BatchSize = 32,
            ShowDownloadProgress = false
        }
    }
};

var result = (await XbergConverter.ExtractAsync(ExtractInput.FromUri("document.pdf"), config)).Results[0];

var chunks = result.Chunks ?? new List<Chunk>();
foreach (var (index, chunk) in chunks.WithIndex())
{
    var chunkId = $"doc_chunk_{index}";
    Console.WriteLine($"Chunk {chunkId}: {chunk.Content[..Math.Min(50, chunk.Content.Length)]}");

    if (chunk.Embedding != null)
    {
        Console.WriteLine($"  Embedding dimensions: {chunk.Embedding.Count}");
    }
}

internal static class EnumerableExtensions
{
    public static IEnumerable<(int Index, T Item)> WithIndex<T>(
        this IEnumerable<T> items)
    {
        var index = 0;
        foreach (var item in items)
        {
            yield return (index++, item);
        }
    }
}

Ruby

Ruby

require 'xberg'

config = Xberg::ExtractionConfig.new(
  chunking: Xberg::ChunkingConfig.new(
    max_characters: 1500,
    overlap: 200,
    embedding: Xberg::EmbeddingConfig.new(
      model: Xberg::EmbeddingModelType.new(
        type: 'preset',
        name: 'text-embedding-all-minilm-l6-v2'
      )
    )
  )
)

Concurrency and Thread Limits

ConcurrencyConfig.max_threads caps every internal thread pool at once: the global Rayon pool, ONNX Runtime intra-op threads, and the combined document/worker budget used by batch extraction.

When max_threads is left unset, the effective budget is min(detected_cpu_cores, 8) — not “use all available cores”. This 8-core ceiling is a deliberate serverless/shared-tenant default, not an auto-scaling target. On a host with more than 8 cores, the extra cores go unused by default:

• Bare metal / VM with no CPU quota: max_threads must be set explicitly above 8 to use more than 8 cores. There is no other way to exceed the default ceiling on this class of host.
• Linux containers under a cgroup CPU quota (e.g. Kubernetes resources.limits.cpu): the quota is used as the ceiling instead of the hardcoded 8, since the quota already reflects a deliberately-configured resource limit. This applies automatically; no configuration is needed.

If none of the above applies and the host has more than 8 cores, a one-time WARN-level log is emitted the first time the thread budget is resolved, naming the detected core count and the applied cap — so the ceiling is discoverable without reading source.

[concurrency]
max_threads = 32

All Configuration Categories

• ExtractionConfig — top-level options
• OcrConfig — OCR backend, language, acceleration
• TesseractConfig — Tesseract PSM, confidence, table detection
• ChunkingConfig — chunk size, overlap
• TokenReductionConfig — LLM prompt token reduction
• ContentFilterConfig — header/footer/watermark filtering
• PageConfig — page tracking and markers
• AccelerationConfig — ONNX Runtime execution provider
• ConcurrencyConfig — thread pool caps (max_threads); not in the auto-generated reference below, see the section above

Next Steps

• Extraction Basics — core extraction API and supported formats
• OCR Guide — backend installation and language setup
• Embeddings — semantic vectors for search
• Language Detection — multilingual document analysis
• Chunking — split text for RAG with page tracking
• Plugins Guide — custom post-processors and validators