Python API Reference

Python API Reference v5.0.0-rc.30¶

Functions¶

extract_bytes()¶

Extract content from a byte array.

This is the main entry point for in-memory extraction. It performs the following steps:

1. Validate MIME type
2. Handle legacy format conversion if needed
3. Select appropriate extractor from registry
4. Extract content
5. Run post-processing pipeline

Returns:

An ExtractionResult containing the extracted content and metadata.

Errors:

Returns KreuzbergError.Validation if MIME type is invalid. Returns KreuzbergError.UnsupportedFormat if MIME type is not supported.

Signature:

def extract_bytes(content: bytes, mime_type: str, config: ExtractionConfig) -> ExtractionResult

Example:

result = extract_bytes(b"data", "value", ExtractionConfig())

Parameters:

Returns: ExtractionResult

Errors: Raises Error.

extract_file()¶

Extract content from a file.

This is the main entry point for file-based extraction. It performs the following steps:

1. Check cache for existing result (if caching enabled)
2. Detect or validate MIME type
3. Select appropriate extractor from registry
4. Extract content
5. Run post-processing pipeline
6. Store result in cache (if caching enabled)

Returns:

An ExtractionResult containing the extracted content and metadata.

Errors:

Returns KreuzbergError.Io if the file doesn't exist (NotFound) or for other file I/O errors. Returns KreuzbergError.UnsupportedFormat if MIME type is not supported.

Signature:

def extract_file(path: str, mime_type: str = None, config: ExtractionConfig) -> ExtractionResult

Example:

result = extract_file("value", mime_type="value", ExtractionConfig())

Parameters:

Returns: ExtractionResult

Errors: Raises Error.

extract_file_sync()¶

Synchronous wrapper for extract_file.

This is a convenience function that blocks the current thread until extraction completes. For async code, use extract_file directly.

Uses the global Tokio runtime for 100x+ performance improvement over creating a new runtime per call. Always uses the global runtime to avoid nested runtime issues.

This function is only available with the tokio-runtime feature. For WASM targets, use a truly synchronous extraction approach instead.

Signature:

def extract_file_sync(path: str, mime_type: str = None, config: ExtractionConfig) -> ExtractionResult

Example:

result = extract_file_sync("value", mime_type="value", ExtractionConfig())

Parameters:

Returns: ExtractionResult

Errors: Raises Error.

extract_bytes_sync()¶

Synchronous wrapper for extract_bytes.

Uses the global Tokio runtime for 100x+ performance improvement over creating a new runtime per call.

With the tokio-runtime feature, this blocks the current thread using the global Tokio runtime. Without it (WASM), this calls a truly synchronous implementation.

Signature:

def extract_bytes_sync(content: bytes, mime_type: str, config: ExtractionConfig) -> ExtractionResult

Example:

result = extract_bytes_sync(b"data", "value", ExtractionConfig())

Parameters:

Returns: ExtractionResult

Errors: Raises Error.

extract_bytes_sync()¶

Synchronous wrapper for extract_bytes (WASM-compatible version).

This is a truly synchronous implementation without tokio runtime dependency. It calls extract_bytes_sync_impl() to perform the extraction.

Signature:

def extract_bytes_sync(content: bytes, mime_type: str, config: ExtractionConfig) -> ExtractionResult

Example:

result = extract_bytes_sync(b"data", "value", ExtractionConfig())

Parameters:

Returns: ExtractionResult

Errors: Raises Error.

batch_extract_files_sync()¶

Synchronous wrapper for batch_extract_files.

Uses the global Tokio runtime for optimal performance. Only available with tokio-runtime (WASM has no filesystem).

Signature:

def batch_extract_files_sync(items: list[BatchFileItem], config: ExtractionConfig) -> list[ExtractionResult]

Example:

result = batch_extract_files_sync([], ExtractionConfig())

Parameters:

Returns: list[ExtractionResult]

Errors: Raises Error.

batch_extract_bytes_sync()¶

Synchronous wrapper for batch_extract_bytes.

Uses the global Tokio runtime for optimal performance. With the tokio-runtime feature, this blocks the current thread using the global Tokio runtime. Without it (WASM), this calls a truly synchronous implementation that iterates through items and calls extract_bytes_sync().

Signature:

def batch_extract_bytes_sync(items: list[BatchBytesItem], config: ExtractionConfig) -> list[ExtractionResult]

Example:

result = batch_extract_bytes_sync([], ExtractionConfig())

Parameters:

Returns: list[ExtractionResult]

Errors: Raises Error.

batch_extract_bytes_sync()¶

Synchronous wrapper for batch_extract_bytes (WASM-compatible version).

Iterates through items sequentially, applying per-file config overrides.

Signature:

def batch_extract_bytes_sync(items: list[BatchBytesItem], config: ExtractionConfig) -> list[ExtractionResult]

Example:

result = batch_extract_bytes_sync([], ExtractionConfig())

Parameters:

Returns: list[ExtractionResult]

Errors: Raises Error.

batch_extract_files()¶

Extract content from multiple files concurrently.

This function processes multiple files in parallel, automatically managing concurrency to prevent resource exhaustion. The concurrency limit can be configured via ExtractionConfig.max_concurrent_extractions or defaults to (num_cpus * 1.5).ceil().

Each file can optionally specify a FileExtractionConfig that overrides specific fields from the batch-level config. Pass None for a file to use the batch defaults. Batch-level settings like max_concurrent_extractions and use_cache are always taken from the batch-level config.

per-file configuration overrides.

• config - Batch-level extraction configuration (provides defaults and batch settings)

Returns:

A vector of ExtractionResult in the same order as the input items.

Errors:

Individual file errors are captured in the result metadata. System errors (IO, RuntimeError equivalents) will bubble up and fail the entire batch.

Simple usage with no per-file overrides:

Per-file configuration overrides:

Signature:

def batch_extract_files(items: list[BatchFileItem], config: ExtractionConfig) -> list[ExtractionResult]

Example:

result = batch_extract_files([], ExtractionConfig())

Parameters:

Returns: list[ExtractionResult]

Errors: Raises Error.

batch_extract_bytes()¶

Extract content from multiple byte arrays concurrently.

This function processes multiple byte arrays in parallel, automatically managing concurrency to prevent resource exhaustion. The concurrency limit can be configured via ExtractionConfig.max_concurrent_extractions or defaults to (num_cpus * 1.5).ceil().

Each item can optionally specify a FileExtractionConfig that overrides specific fields from the batch-level config. Pass None as the config to use the batch-level defaults for that item.

MIME type, and optional per-item configuration overrides.

• config - Batch-level extraction configuration

Returns:

A vector of ExtractionResult in the same order as the input items.

Simple usage with no per-item overrides:

Per-item configuration overrides:

Signature:

def batch_extract_bytes(items: list[BatchBytesItem], config: ExtractionConfig) -> list[ExtractionResult]

Example:

result = batch_extract_bytes([], ExtractionConfig())

Parameters:

Returns: list[ExtractionResult]

Errors: Raises Error.

detect_mime_type_from_bytes()¶

Detect MIME type from raw file bytes.

Uses magic byte signatures to detect file type from content. Falls back to infer crate for comprehensive detection.

For ZIP-based files, inspects contents to distinguish Office Open XML formats (DOCX, XLSX, PPTX) from plain ZIP archives.

Returns:

The detected MIME type string.

Errors:

Returns KreuzbergError.UnsupportedFormat if MIME type cannot be determined.

Signature:

def detect_mime_type_from_bytes(content: bytes) -> str

Example:

result = detect_mime_type_from_bytes(b"data")

Parameters:

Returns: str

Errors: Raises Error.

get_extensions_for_mime()¶

Get file extensions for a given MIME type.

Returns all known file extensions that map to the specified MIME type.

Returns:

A vector of file extensions (without leading dot) for the MIME type.

Signature:

def get_extensions_for_mime(mime_type: str) -> list[str]

Example:

result = get_extensions_for_mime("value")

Parameters:

Returns: list[str]

Errors: Raises Error.

list_supported_formats()¶

List all supported document formats.

Returns every file extension Kreuzberg recognizes together with its corresponding MIME type, derived from the central format registry. Formats that have no registered file extension (such as source code, which is detected dynamically) are not included.

The list is sorted alphabetically by file extension.

Returns:

A vector of SupportedFormat entries sorted by extension.

Signature:

def list_supported_formats() -> list[SupportedFormat]

Example:

result = list_supported_formats()

Returns: list[SupportedFormat]

detect_qr_codes()¶

Detect QR codes in the bytes of an ExtractedImage.

format_hint is currently unused — the image crate auto-detects the container format from magic bytes — but the parameter is retained so future backends (e.g. a WebP-via-webp-decoder variant) can use it without an API break.

Returns an empty listtor on any of:

• Empty input.
• Image-decode failure.
• No QR grids detected.
• All detected grids fail to decode.

Successfully decoded QR codes carry their payload, a confidence of 1.0 (rqrr does not expose per-grid confidence; a successful decode is treated as high-confidence by convention), and the pixel-space bounding box derived from the four corner points of the grid.

Signature:

def detect_qr_codes(image_bytes: bytes, format_hint: str = None) -> list[QrCode]

Example:

result = detect_qr_codes(b"data", format_hint="value")

Parameters:

Returns: list[QrCode]

clear_embedding_backends()¶

Clear all embedding backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

• Any error returned by a backend's shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

def clear_embedding_backends() -> None

Example:

clear_embedding_backends()

Returns: No return value.

Errors: Raises Error.

list_embedding_backends()¶

List the names of all registered embedding backends.

Used by kreuzberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

def list_embedding_backends() -> list[str]

Example:

result = list_embedding_backends()

Returns: list[str]

Errors: Raises Error.

list_document_extractors()¶

List names of all registered document extractors.

Signature:

def list_document_extractors() -> list[str]

Example:

result = list_document_extractors()

Returns: list[str]

Errors: Raises Error.

clear_document_extractors()¶

Clear all document extractors from the global registry.

Calls shutdown() on every registered extractor, then empties the registry.

Errors:

• Any error returned by an extractor's shutdown() method. The first error encountered stops processing of remaining extractors.

Signature:

def clear_document_extractors() -> None

Example:

clear_document_extractors()

Returns: No return value.

Errors: Raises Error.

list_ocr_backends()¶

List all registered OCR backends.

Returns the names of all OCR backends currently registered in the global registry.

Returns:

A vector of OCR backend names.

Signature:

def list_ocr_backends() -> list[str]

Example:

result = list_ocr_backends()

Returns: list[str]

Errors: Raises Error.

clear_ocr_backends()¶

Clear all OCR backends from the global registry.

Removes all OCR backends and calls their shutdown() methods.

Returns:

• Ok(()) if all backends were cleared successfully
• Err(...) if any shutdown method failed

Signature:

def clear_ocr_backends() -> None

Example:

clear_ocr_backends()

Returns: No return value.

Errors: Raises Error.

register_builtin()¶

Register every built-in post-processor enabled by the active feature set.

This is the single entry point that callers (including register_default_post_processors) use to populate the global post-processor registry with the in-tree built-ins. Each submodule's own register function is gated by its feature flag so this aggregate stays safe to call on any target.

Signature:

def register_builtin() -> None

Example:

register_builtin()

Returns: No return value.

Errors: Raises Error.

list_post_processors()¶

List all registered post-processor names.

Returns a vector of all post-processor names currently registered in the global registry.

Returns:

• Ok(list[str]) - Vector of post-processor names
• Err(...) if the registry lock is poisoned

Signature:

def list_post_processors() -> list[str]

Example:

result = list_post_processors()

Returns: list[str]

Errors: Raises Error.

clear_post_processors()¶

Remove all registered post-processors.

Signature:

def clear_post_processors() -> None

Example:

clear_post_processors()

Returns: No return value.

Errors: Raises Error.

list_renderers()¶

List names of all registered renderers.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

def list_renderers() -> list[str]

Example:

result = list_renderers()

Returns: list[str]

Errors: Raises Error.

clear_renderers()¶

Clear all renderers from the global registry.

Removes every renderer, including the built-in defaults (markdown, html, djot, plain). After calling this no renderers are registered; re-register as needed.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

def clear_renderers() -> None

Example:

clear_renderers()

Returns: No return value.

Errors: Raises Error.

clear_reranker_backends()¶

Clear all reranker backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

• Any error returned by a backend's shutdown() method. The first error encountered stops processing of remaining backends.

Since v5.0.

Signature:

def clear_reranker_backends() -> None

Example:

clear_reranker_backends()

Returns: No return value.

Errors: Raises Error.

list_reranker_backends()¶

List the names of all registered reranker backends.

Used by kreuzberg-cli, the api/mcp endpoints, and generated language bindings.

Since v5.0.

Signature:

def list_reranker_backends() -> list[str]

Example:

result = list_reranker_backends()

Returns: list[str]

Errors: Raises Error.

list_validators()¶

List names of all registered validators.

Signature:

def list_validators() -> list[str]

Example:

result = list_validators()

Returns: list[str]

Errors: Raises Error.

clear_validators()¶

Remove all registered validators.

Signature:

def clear_validators() -> None

Example:

clear_validators()

Returns: No return value.

Errors: Raises Error.

classify_pages()¶

Run page classification against an extraction result.

Mutates result.page_classifications with one entry per non-empty page and appends every LLM call's usage to result.llm_usage.

Errors:

Returns the first error encountered when rendering the prompt or calling the LLM. Partially produced classifications are discarded so callers do not see a half-populated vector.

Signature:

def classify_pages(result: ExtractionResult, config: PageClassificationConfig) -> None

Example:

classify_pages(ExtractionResult(), PageClassificationConfig())

Parameters:

Returns: No return value.

Errors: Raises Error.

classify_text()¶

Classify a single piece of text without requiring an ExtractionResult.

Use this when the caller already has plain text (e.g. a RAG ingest pipeline receiving documents off a queue) and wants a label list back without manufacturing extractor-side metadata.

Errors:

Same as classify_pages: a validation error when config.labels is empty, or any error returned by prompt rendering or the underlying LLM call.

Signature:

def classify_text(text: str, config: PageClassificationConfig) -> list[ClassificationLabel]

Example:

result = classify_text("value", PageClassificationConfig())

Parameters:

Returns: list[ClassificationLabel]

Errors: Raises Error.

classify_document()¶

Classify a single document (as multiple pages or a single text block).

Aggregates classifications across all pages in the provided text, returning a combined label set that represents the document as a whole.

using the configured LLM, and results are aggregated.

• config - Classification configuration including labels and LLM settings.

Returns:

A vector of ClassificationLabel entries representing the document's overall classification.

Errors:

Returns an error if config.labels is empty or if LLM calls fail.

Signature:

def classify_document(pages: list[str], config: PageClassificationConfig) -> list[ClassificationLabel]

Example:

result = classify_document([], PageClassificationConfig())

Parameters:

Returns: list[ClassificationLabel]

Errors: Raises Error.

download_model()¶

Eagerly download a NER model into the kreuzberg cache.

name is a HuggingFace repo id (e.g. urchade/gliner_multi-v2.1). The CLI flag kreuzberg warm --ner delegates here.

Signature:

def download_model(name: str, cache_dir: str = None) -> str

Example:

result = download_model("value", cache_dir="value")

Parameters:

Returns: str

Errors: Raises Error.

download_model()¶

Signature:

def download_model(name: str, cache_dir: str = None) -> str

Example:

result = download_model("value", cache_dir="value")

Parameters:

Returns: str

Errors: Raises Error.

default_model_name()¶

Pinned default NER model identifier.

Signature:

def default_model_name() -> str

Example:

result = default_model_name()

Returns: str

default_model_name()¶

Signature:

def default_model_name() -> str

Example:

result = default_model_name()

Returns: str

known_models()¶

All NER models kreuzberg knows about (used by --all-ner-models).

Signature:

def known_models() -> list[str]

Example:

result = known_models()

Returns: list[str]

known_models()¶

Signature:

def known_models() -> list[str]

Example:

result = known_models()

Returns: list[str]

download_model()¶

Download a NER model into the kreuzberg cache.

Signature:

def download_model(name: str, cache_dir: str = None) -> str

Example:

result = download_model("value", cache_dir="value")

Parameters:

Returns: str

Errors: Raises Error.

default_model_name()¶

Default NER model identifier.

Signature:

def default_model_name() -> str

Example:

result = default_model_name()

Returns: str

known_models()¶

All NER models kreuzberg knows about.

Signature:

def known_models() -> list[str]

Example:

result = known_models()

Returns: list[str]

redact()¶

Run pattern redaction (and optional NER-driven redaction) over result and rewrite every textual field. Populates result.redaction_report.

Signature:

def redact(result: ExtractionResult, config: RedactionConfig) -> None

Example:

redact(ExtractionResult(), RedactionConfig())

Parameters:

Returns: No return value.

Errors: Raises Error.

find_all()¶

Find all US Social Security Number spans in text (format: NNN-NN-NNNN).

Signature:

def find_all(text: str) -> list[PatternMatch]

Example:

result = find_all("value")

Parameters:

Returns: list[PatternMatch]

scan_text()¶

Scan text for every PII category in categories and return all matches in source-byte order.

When categories is empty every supported regex-detectable category fires. Person / Organization / Location are not covered by the pattern engine — they must be supplied by a NER backend through the redaction engine.

Signature:

def scan_text(text: str, categories: list[PiiCategory]) -> list[PatternMatch]

Example:

result = scan_text("value", [])

Parameters:

Returns: list[PatternMatch]

summarize()¶

Score and return the top-N sentences from text, joined in original order.

language is an ISO 639 (or locale) code used to pick a stopword list; pass None (or an unknown code) to fall back to English. max_tokens bounds the summary length by whitespace-separated tokens; None falls back to DEFAULT_MAX_TOKENS.

Signature:

def summarize(text: str, language: str = None, max_tokens: int = None) -> str | None

Example:

result = summarize("value", language="value", max_tokens=42)

Parameters:

Returns: str | None

token_count()¶

Count whitespace-separated tokens (used for token-budget bookkeeping by callers).

Signature:

def token_count(text: str) -> int

Example:

result = token_count("value")

Parameters:

Returns: int

translate_result()¶

Translate the extraction result in place.

Populates result.translation with the translated content, optionally the translated formatted_content (when preserve_markup = true), and rewrites every chunk's content field. Every LLM call's usage is appended to result.llm_usage.

Signature:

def translate_result(result: ExtractionResult, config: TranslationConfig) -> None

Example:

translate_result(ExtractionResult(), TranslationConfig())

Parameters:

Returns: No return value.

Errors: Raises Error.

find_footnote_anchors()¶

Find all footnote anchor references in markdown text.

Returns a vector of footnote anchors ([^label] use-sites), including byte offsets. Footnote definitions ([^label]: ...) are NOT included in the results.

Returns:

A vector of FootnoteAnchor entries, each with the label and byte offset.

Signature:

def find_footnote_anchors(markdown: str) -> list[FootnoteAnchor]

Example:

result = find_footnote_anchors("value")

Parameters:

Returns: list[FootnoteAnchor]

parse_footnote_definitions()¶

Parse footnote definitions from markdown text.

Returns a vector of footnote definitions found in the markdown. Handles multi-line definitions with continuation/indented lines (CommonMark format).

Returns:

A vector of FootnoteDefinition entries, each with label, content, and byte offset.

Signature:

def parse_footnote_definitions(markdown: str) -> list[FootnoteDefinition]

Example:

result = parse_footnote_definitions("value")

Parameters:

Returns: list[FootnoteDefinition]

find_inference_markers()¶

Find inference markers in markdown text.

Returns byte offsets of every [*inference*] marker found in the text.

Returns:

A vector of byte offsets where inference markers appear.

Signature:

def find_inference_markers(markdown: str) -> list[int]

Example:

result = find_inference_markers("value")

Parameters:

Returns: list[int]

find_unmarked_claims()¶

Find unmarked claims in markdown text.

Returns lines that assert a claim but carry neither a footnote citation anchor ([^...]) nor an inference marker ([*inference*]).

The heuristic is simple: a line that contains alphabetic words, ends with sentence punctuation, and is not a heading, blank line, or markup-only line is considered a claim. Exclude lines that appear in the citation block (after --- + <!-- citations ... -->).

Returns:

A vector of trimmed line text strings for unmarked claims.

Signature:

def find_unmarked_claims(markdown: str) -> list[str]

Example:

result = find_unmarked_claims("value")

Parameters:

Returns: list[str]

parse_citations()¶

Parse the structured citation block from markdown.

Extracts citations from the block after a --- thematic break followed by <!-- citations ... --> comment. Parses each entry as: [^srcN]: <source>, <optional-locator>, excerpt: "<text>"

Returns parsed citations with source, optional locator, and optional excerpt.

Returns:

A vector of Citation entries parsed from the citation block.

Signature:

def parse_citations(markdown: str) -> list[Citation]

Example:

result = parse_citations("value")

Parameters:

Returns: list[Citation]

verify_excerpt()¶

Verify that an excerpt appears verbatim in source text.

Performs exact matching by default. Also tries whitespace-normalized matching (collapsing runs of whitespace on both sides) since PDF-extracted text often has irregular spacing.

Returns:

True if the excerpt appears (exactly or with normalized whitespace), False otherwise.

Signature:

def verify_excerpt(excerpt: str, source_text: str) -> bool

Example:

result = verify_excerpt("value", "value")

Parameters:

Returns: bool

chunk_for_rag()¶

Chunk text for RAG retrieval, ensuring every chunk carries a heading_path.

Delegates to chunk_text using the caller's config (defaulting to ChunkerType.Markdown when the config uses the default Text type, so that heading hierarchy is resolved). After chunking, derives ChunkMetadata.heading_path from each chunk's heading_context.

underlying splitter; use ChunkerType.Markdown for documents with ATX headings.

Returns:

A ChunkingResult where every chunk's heading_path is populated from its heading_context (empty when the chunk is not under any heading).

Errors:

Propagates any error from the underlying chunker (e.g. invalid overlap).

Signature:

def chunk_for_rag(text: str, config: ChunkingConfig) -> ChunkingResult

Example:

result = chunk_for_rag("value", ChunkingConfig())

Parameters:

Returns: ChunkingResult

Errors: Raises Error.

compare()¶

Compare two extraction results and return a structured diff.

The comparison is purely structural — no I/O, no side effects. All fields of ExtractionDiff are populated according to the provided DiffOptions.

Signature:

def compare(a: ExtractionResult, b: ExtractionResult, opts: DiffOptions) -> ExtractionDiff

Example:

result = compare(ExtractionResult(), ExtractionResult(), DiffOptions())

Parameters:

Returns: ExtractionDiff

extract_region_with_vlm()¶

Extract content from a pre-cropped image region using a VLM.

The caller is responsible for cropping the page image to the region's bounding box before calling this function. The image_bytes parameter must contain the raw bytes of the cropped region image (JPEG, PNG, WebP, etc.).

Returns:

Extracted Markdown text from the VLM, or an error if the VLM call fails.

Errors:

• Ocr if the VLM call fails or returns no content.
• MissingDependency if the liter-llm client cannot be initialised.

Signature:

def extract_region_with_vlm(image_bytes: bytes, image_mime: str, region_kind: RegionKind, llm_config: LlmConfig, custom_prompt: str = None) -> str

Example:

result = extract_region_with_vlm(b"data", "value", RegionKind(), LlmConfig(), custom_prompt="value")

Parameters:

Returns: str

Errors: Raises Error.

rerank_async()¶

Rerank documents asynchronously.

Async counterpart to rerank. Offloads blocking ONNX inference to a dedicated blocking thread pool via Tokio's spawn_blocking, keeping the async executor free.

Since v5.0.

Signature:

def rerank_async(query: str, documents: list[str], config: RerankerConfig) -> list[RerankedDocument]

Example:

result = rerank_async("value", [], RerankerConfig())

Parameters:

Returns: list[RerankedDocument]

Errors: Raises Error.

extract_keywords()¶

Extract keywords from text using the specified algorithm.

This is the unified entry point for keyword extraction. The algorithm used is determined by config.algorithm.

Returns:

A vector of keywords sorted by relevance (highest score first).

Errors:

Returns an error if:

• The specified algorithm feature is not enabled
• Keyword extraction fails

Signature:

def extract_keywords(text: str, config: KeywordConfig) -> list[Keyword]

Example:

result = extract_keywords("value", KeywordConfig())

Parameters:

Returns: list[Keyword]

Errors: Raises Error.

analyze_document()¶

Analyze a document and determine the optimal chunking strategy.

Decision logic (in priority order):

1. If user provides disable_chunking → no chunking
2. If user provides page_ranges → use user overrides
3. If chunking is not enabled → no chunking
4. If format doesn't support chunking → no chunking
5. If file is small (below both thresholds) and not force_chunking → no chunking

6. If PDF has a substantial text layer AND !force_ocr → no chunking (only when heuristics-pdf feature is enabled; otherwise skipped)

7. Otherwise → chunk the document

Errors:

Returns an error only when the heuristics-pdf feature is active and the PDF text-layer analysis itself returns a hard error. In all other cases the function returns a ChunkingDecision.

Signature:

def analyze_document(metadata: DocumentMetadata, config: HeuristicsConfig, document_bytes: bytes = None) -> ChunkingDecision

Example:

result = analyze_document(DocumentMetadata(), HeuristicsConfig(), document_bytes=b"data")

Parameters:

Returns: ChunkingDecision

Errors: Raises Error.

analyze_with_user_chunks()¶

Analyze a document with user-specified chunk ranges.

Creates a chunk plan based on user-provided page ranges.

Signature:

def analyze_with_user_chunks(user_ranges: list[PageRange], total_pages: int, size_bytes: int, config: HeuristicsConfig) -> ChunkingDecision

Example:

result = analyze_with_user_chunks([], 42, 42, HeuristicsConfig())

Parameters:

Returns: ChunkingDecision

score_confidence()¶

Score a ConfidenceSignals triple into an ExtractionConfidence using the supplied weights.

When signals.ocr_aggregate is None, the OCR weight folds into text_coverage so the weighted sum still totals 1.0.

Signature:

def score_confidence(signals: ConfidenceSignals, weights: ConfidenceWeights) -> ExtractionConfidence

Example:

result = score_confidence(ConfidenceSignals(), ConfidenceWeights())

Parameters:

Returns: ExtractionConfidence

check_format_limits()¶

Decision returned for pre-extraction rejection based on XLSX/PPTX-specific resource bounds. Returns Some(reason) to reject; None to proceed.

Callers must provide counts from a pre-extraction peek (e.g. parsing xl/workbook.xml for sheet count).

Signature:

def check_format_limits(mime_type: str, sheet_count: int = None, workbook_cells: int = None, embedded_count: int = None, config: HeuristicsConfig) -> str | None

Example:

result = check_format_limits("value", sheet_count=42, workbook_cells=42, embedded_count=42, HeuristicsConfig())

Parameters:

Returns: str | None

boundaries_from_extraction_result()¶

Derive document boundaries from an already-produced ExtractionResult.

Builds a MultidocInput from result.pages (one PageSignals per PageContent entry), then delegates to detect_boundaries.

Fallback behaviour¶

• If result.pages is None or empty the whole document is treated as a single document: returns [Start(1), End(1)], matching the contract of detect_boundaries for a one-page input.

Text density¶

PageContent does not carry a pre-computed density score. This function approximates density as non_whitespace_chars / total_chars (clamped to [0.0, 1.0]), which is a reasonable proxy for how text-dense a page is relative to itself. Pass a custom MultidocInput to detect_boundaries directly when you need a higher-fidelity density measurement (e.g. chars-per-pt² from a PDF extractor).

Signature:

def boundaries_from_extraction_result(result: ExtractionResult, thresholds: MultidocThresholds) -> list[DocumentBoundary]

Example:

result = boundaries_from_extraction_result(ExtractionResult(), MultidocThresholds())

Parameters:

Returns: list[DocumentBoundary]

detect_boundaries()¶

Detect document boundaries in a multi-document PDF.

Returns a list of detected boundaries, always including implicit boundaries at start (page 1) and end (page_count). Boundaries are returned in ascending order of start_page.

Returns:

Ordered list of document boundaries.

Signature:

def detect_boundaries(input: MultidocInput, thresholds: MultidocThresholds) -> list[DocumentBoundary]

Example:

result = detect_boundaries(MultidocInput(), MultidocThresholds())

Parameters:

Returns: list[DocumentBoundary]

choose_call_mode()¶

Decide which call mode best fits this document.

Rules applied in order:

1. image/* → StructuredCallMode.VisionOnly (no text layer to start from).

2. application/pdf → StructuredCallMode.TextOnly regardless of text_coverage or embedded image count. Kreuzberg's OCR + text-layer extraction produces text for scanned PDFs; the orchestrator's post-call confidence gate handles any vision escalation actually needed.

3. DOCX / text/html / text/* / application/json / application/xml / application/rtf with avg_chars_per_page > docx_text_min_density → StructuredCallMode.TextOnly.

4. Anything else → StructuredCallMode.Skip.

After rule selection two post-rule promotions apply (in order):

• user_force_vision promotes TextOnly → TextPlusVision (Skip stays Skip — caller meant to opt out).

• enable_vision_fallback promotes TextOnly → TextOnlyWithVisionFallback (does not upgrade TextPlusVision or Skip).

Signature:

def choose_call_mode(input: StructuredInput, t: StructuredThresholds) -> StructuredCallMode

Example:

result = choose_call_mode(StructuredInput(), StructuredThresholds())

Parameters:

Returns: StructuredCallMode

calculate_chunk_plan()¶

Calculate a chunking plan for a document.

Returns:

A ChunkPlan with optimal chunk boundaries.

Signature:

def calculate_chunk_plan(page_count: int, size_bytes: int, needs_ocr: bool, config: HeuristicsConfig) -> ChunkPlan

Example:

result = calculate_chunk_plan(42, 42, True, HeuristicsConfig())

Parameters:

Returns: ChunkPlan

calculate_plan_from_overrides()¶

Calculate a chunk plan from user-specified page ranges.

Validates and processes user overrides into a proper chunk plan.

Signature:

def calculate_plan_from_overrides(user_chunks: list[PageRange], total_pages: int, size_bytes: int, config: HeuristicsConfig) -> ChunkPlan

Example:

result = calculate_plan_from_overrides([], 42, 42, HeuristicsConfig())

Parameters:

Returns: ChunkPlan

fingerprint()¶

Stable sha256 fingerprint of raw, formatted as sha256:<hex>.

Signature:

def fingerprint(raw: bytes) -> str

Example:

result = fingerprint(b"data")

Parameters:

Returns: str

resolve()¶

Resolve (preset, custom_schema_override, context) into a ResolvedPreset.

• custom_schema overrides preset.schema when set.
• context substitutes {{key}} tokens in preset.context_template; the rendered string is appended to system_prompt so the model sees it.

Signature:

def resolve(preset: Preset, custom_schema: dict[str, Any] = None, context: dict[str, str]) -> ResolvedPreset

Example:

result = resolve(Preset(), custom_schema={}, {})

Parameters:

Returns: ResolvedPreset

Errors: Raises ResolveError.

extract_structured_json()¶

Extract structured JSON from a document using JSON-encoded preset spec and options.

This is the synchronous JSON-in / JSON-out entry point suitable for FFI and language-binding call paths.

cache). Pass "{}" to use all defaults.

Returns:

JSON-serialised StructuredOutput on success.

Errors:

Returns Validation when either JSON argument is malformed. All other failures from the underlying extract_structured_sync call are mapped onto KreuzbergError via From<StructuredError>.

Signature:

def extract_structured_json(bytes: bytes, mime: str, preset_spec_json: str, options_json: str) -> str

Example:

result = extract_structured_json(b"data", "value", "value", "value")

Parameters:

Returns: str

Errors: Raises Error.

split_and_extract_json()¶

Split a multi-document PDF and extract structured JSON from each segment, returning a JSON array of StructuredOutput objects.

Non-PDF documents are passed through as a single-element array.

Same as extract_structured_json.

Returns:

JSON-serialised list[StructuredOutput] (a JSON array) on success.

Errors:

Returns Validation when either JSON argument is malformed. All other failures from the underlying split_and_extract_sync call are mapped onto KreuzbergError via From<StructuredError>.

Signature:

def split_and_extract_json(bytes: bytes, mime: str, preset_spec_json: str, options_json: str) -> str

Example:

result = split_and_extract_json(b"data", "value", "value", "value")

Parameters:

Returns: str

Errors: Raises Error.

render_pdf_page_to_png()¶

Render a single PDF page to PNG bytes.

Returns raw PNG-encoded bytes for the specified page at the given DPI. Uses pdf_oxide with tiny-skia for pure-Rust rendering.

For pages with extreme dimensions (very wide vector diagrams, etc.) the effective DPI may be automatically reduced to avoid rasterizer failure. A warning is logged when this happens.

Errors:

Returns KreuzbergError.Parsing if the PDF cannot be opened, authenticated, or rendered, or if page_index is out of range.

Signature:

def render_pdf_page_to_png(pdf_bytes: bytes, page_index: int, dpi: int = None, password: str = None) -> bytes

Example:

result = render_pdf_page_to_png(b"data", 42, dpi=42, password="value")

Parameters:

Returns: bytes

Errors: Raises Error.

caption_image()¶

Caption a single image from bytes.

RegionKind.Caption prompt when None.

Returns:

The generated caption text.

Errors:

Returns an error if the VLM call fails or if image format detection fails.

Signature:

def caption_image(image_bytes: bytes, llm_config: LlmConfig, custom_prompt: str = None) -> str

Example:

result = caption_image(b"data", LlmConfig(), custom_prompt="value")

Parameters:

Returns: str

Errors: Raises Error.

caption_image_file()¶

Caption a single image from a file path.

RegionKind.Caption prompt when None.

Returns:

The generated caption text.

Errors:

Returns an error if the file cannot be read, if image format detection fails, or if the VLM call fails.

Signature:

def caption_image_file(path: str, llm_config: LlmConfig, custom_prompt: str = None) -> str

Example:

result = caption_image_file("value", LlmConfig(), custom_prompt="value")

Parameters:

Returns: str

Errors: Raises Error.

detect_mime_type()¶

Detect the MIME type of a file at the given path.

Uses the file extension and optionally the file content to determine the MIME type. Set check_exists to True to verify the file exists before detection.

Signature:

def detect_mime_type(path: str, check_exists: bool) -> str

Example:

result = detect_mime_type("value", True)

Parameters:

Returns: str

Errors: Raises Error.

embed_texts_async()¶

Signature:

def embed_texts_async(texts: list[str], config: EmbeddingConfig) -> list[list[float]]

Example:

result = embed_texts_async([], EmbeddingConfig())

Parameters:

Returns: list[list[float]]

Errors: Raises Error.

get_embedding_preset()¶

Get an embedding preset by name.

Returns None if no preset with the given name exists. Returns an owned clone so the value is safe to pass across FFI boundaries.

Signature:

def get_embedding_preset(name: str) -> EmbeddingPreset | None

Example:

result = get_embedding_preset("value")

Parameters:

Returns: EmbeddingPreset | None

list_embedding_presets()¶

List the names of all available embedding presets.

Returns owned Strings so the values are safe to pass across FFI boundaries.

Signature:

def list_embedding_presets() -> list[str]

Example:

result = list_embedding_presets()

Returns: list[str]

get_embedding_preset()¶

Returns None for builds without the embedding-presets feature.

Signature:

def get_embedding_preset(name: str) -> EmbeddingPreset | None

Example:

result = get_embedding_preset("value")

Parameters:

Returns: EmbeddingPreset | None

list_embedding_presets()¶

Returns an empty list for builds without the embedding-presets feature.

Signature:

def list_embedding_presets() -> list[str]

Example:

result = list_embedding_presets()

Returns: list[str]

rerank()¶

Rerank a list of documents by relevance to a query.

Returns documents sorted descending by score. Applies top_k truncation if configured.

Errors:

• KreuzbergError.Validation if query is empty or blank.
• KreuzbergError.MissingDependency if ONNX Runtime is not installed (ONNX path).
• KreuzbergError.Reranking if the preset is unknown or model download fails.

Since v5.0.

Signature:

def rerank(query: str, documents: list[str], config: RerankerConfig) -> list[RerankedDocument]

Example:

result = rerank("value", [], RerankerConfig())

Parameters:

Returns: list[RerankedDocument]

Errors: Raises Error.

rerank()¶

Stub for builds without the reranker feature — keeps the symbol available on no-ORT targets (Android x86_64 emulator, WASM) so language bindings compile.

Since v5.0.

Signature:

def rerank(query: str, documents: list[str], config: RerankerConfig) -> list[RerankedDocument]

Example:

result = rerank("value", [], RerankerConfig())

Parameters:

Returns: list[RerankedDocument]

Errors: Raises Error.

rerank_async()¶

Stub for builds without the reranker feature.

Since v5.0.

Signature:

def rerank_async(query: str, documents: list[str], config: RerankerConfig) -> list[RerankedDocument]

Example:

result = rerank_async("value", [], RerankerConfig())

Parameters:

Returns: list[RerankedDocument]

Errors: Raises Error.

get_reranker_preset()¶

Get a reranker preset by name.

Returns None if no preset with the given name exists. Returns an owned clone so the value is safe to pass across FFI boundaries.

Since v5.0.

Signature:

def get_reranker_preset(name: str) -> RerankerPreset | None

Example:

result = get_reranker_preset("value")

Parameters:

Returns: RerankerPreset | None

list_reranker_presets()¶

List the names of all available reranker presets.

Returns owned Strings so the values are safe to pass across FFI boundaries.

Since v5.0.

Signature:

def list_reranker_presets() -> list[str]

Example:

result = list_reranker_presets()

Returns: list[str]

get_reranker_preset()¶

Returns None for builds without the reranker-presets feature.

Since v5.0.

Signature:

def get_reranker_preset(name: str) -> RerankerPreset | None

Example:

result = get_reranker_preset("value")

Parameters:

Returns: RerankerPreset | None

list_reranker_presets()¶

Returns an empty list for builds without the reranker-presets feature.

Since v5.0.

Signature:

def list_reranker_presets() -> list[str]

Example:

result = list_reranker_presets()

Returns: list[str]

embed_texts_async()¶

Signature:

def embed_texts_async(texts: list[str], config: EmbeddingConfig) -> list[list[float]]

Example:

result = embed_texts_async([], EmbeddingConfig())

Parameters:

Returns: list[list[float]]

Errors: Raises Error.

Types¶

AccelerationConfig¶

Hardware acceleration configuration for ONNX Runtime models.

Controls which execution provider (CPU, CoreML, CUDA, TensorRT) is used for inference in layout detection and embedding generation.

ArchiveEntry¶

A single file extracted from an archive.

When archives (ZIP, TAR, 7Z, GZIP) are extracted with recursive extraction enabled, each processable file produces its own full ExtractionResult.

ArchiveMetadata¶

Archive (ZIP/TAR/7Z) metadata.

Extracted from compressed archive files containing file lists and size information.

AudioMetadata¶

Audio/video file metadata.

Populated from container tags (ID3v2, MP4 atoms, Vorbis comments, etc.) and PCM decode properties. Available when the transcription-types feature is enabled.

BBox¶

Bounding box in original image coordinates (x1, y1) top-left, (x2, y2) bottom-right.

BatchBytesItem¶

Batch item for byte array extraction.

Used with batch_extract_bytes and batch_extract_bytes_sync to represent a single item in a batch extraction job.

BatchFileItem¶

Batch item for file extraction.

Used with batch_extract_files and batch_extract_files_sync to represent a single file in a batch extraction job.

BibtexMetadata¶

BibTeX bibliography metadata.

BoundingBox¶

Bounding box coordinates for element positioning.

CacheStats¶

Aggregate statistics for a kreuzberg cache directory.

CaptioningConfig¶

Since: v5.0

Configuration for the VLM captioning post-processor.

CaptioningEnrichmentConfig¶

Captioning enrichment knob: which LLM to use for image captions.

The enrichment stage calls caption_image for every image in ExtractionResult.images that has non-empty data. Images with empty byte data (e.g. reference-only images populated via source_path) are skipped rather than forwarded to the VLM.

CellChange¶

A single changed cell within a table.

Defined here (rather than only in crate.diff) so RevisionDelta can reference it unconditionally, without requiring the diff Cargo feature. crate.diff re-exports this type verbatim.

Chunk¶

A text chunk with optional embedding and metadata.

Chunks are created when chunking is enabled in ExtractionConfig. Each chunk contains the text content, optional embedding vector (if embedding generation is configured), and metadata about its position in the document.

ChunkInfo¶

Information about a single chunk.

ChunkMetadata¶

Metadata about a chunk's position in the original document.

ChunkPlan¶

Complete chunking plan for a document.

Methods¶

default()¶

An empty plan (no chunks). The reason is a placeholder since an empty plan has no chunking rationale; callers always overwrite it when a real plan is built.

Signature:

@staticmethod
def default() -> ChunkPlan

Example:

result = ChunkPlan.default()

Returns: ChunkPlan

total_pages()¶

Get the total number of pages across all chunks.

Signature:

def total_pages(self) -> int

Example:

result = instance.total_pages()

Returns: int

ChunkingConfig¶

Chunking configuration.

Configures text chunking for document content, including chunk size, overlap, trimming behavior, and optional embeddings.

Use ..the default constructor when constructing to allow for future field additions:

Methods¶

default()¶

Signature:

@staticmethod
def default() -> ChunkingConfig

Example:

result = ChunkingConfig.default()

Returns: ChunkingConfig

ChunkingResult¶

Result of a text chunking operation.

Contains the generated chunks and metadata about the chunking.

Citation¶

A structured citation from a citation block.

Parsed from entries like: [^srcN]: source, locator, excerpt: "text"

CitationMetadata¶

Citation file metadata (RIS, PubMed, EndNote).

ClassificationEnrichmentConfig¶

Classification enrichment knob: how to label the document.

ClassificationLabel¶

A single label + confidence pair.

ConfidenceSignals¶

Input signals for confidence scoring.

Caller fills these from the extraction result and the LLM response.

Methods¶

from_extraction_result()¶

Build ConfidenceSignals from an ExtractionResult.

• result — The extraction result whose ocr_elements are inspected.
• schema_compliance — Caller-supplied schema validation outcome.
• text_coverage — Caller-supplied fraction of pages with usable text (e.g. 1.0 for native text formats, value from PDF analysis for PDFs).

The ocr_aggregate is computed as the arithmetic mean of all ocr_elements[].confidence.recognition values. When ocr_elements is None or empty the field is set to None.

Signature:

@staticmethod
def from_extraction_result(result: ExtractionResult, schema_compliance: SchemaCompliance, text_coverage: float) -> ConfidenceSignals

Example:

result = ConfidenceSignals.from_extraction_result(ExtractionResult(), SchemaCompliance(), 0.5)

Parameters:

Returns: ConfidenceSignals

ConfidenceWeights¶

Tunable weights for the confidence scoring formula.

Defaults picked by inspection; callers tune them via config.

Methods¶

default()¶

Signature:

@staticmethod
def default() -> ConfidenceWeights

Example:

result = ConfidenceWeights.default()

Returns: ConfidenceWeights

is_normalized()¶

Validate that weights sum to approximately 1.0.

Signature:

def is_normalized(self) -> bool

Example:

result = instance.is_normalized()

Returns: bool

ContentFilterConfig¶

Cross-extractor content filtering configuration.

Controls whether "furniture" content (headers, footers, page numbers, watermarks, repeating text) is included in or stripped from extraction results. Applies across all extractors (PDF, DOCX, RTF, ODT, HTML, etc.) with format-specific implementation.

When None on ExtractionConfig, each extractor uses its current default behavior unchanged.

Methods¶

default()¶

Signature:

@staticmethod
def default() -> ContentFilterConfig

Example:

result = ContentFilterConfig.default()

Returns: ContentFilterConfig

ContributorRole¶

JATS contributor with role.

CoreProperties¶

Dublin Core metadata from docProps/core.xml

Contains standard metadata fields defined by the Dublin Core standard and Office-specific extensions.

CsvMetadata¶

CSV/TSV file metadata.