Rust API Reference

Rust API Reference v1.3.8

Functions

embedded_build_id()

Return the source build identifier embedded in this Xberg library.

This is intended for executable provenance checks, not as a stable content or cache identifier.

Signature:

pub fn embedded_build_id() -> String

Example:

let result = embedded_build_id();

Returns: String

get_cache_metadata()

Return aggregate statistics for the cache directory at cache_dir.

Errors:

Returns an error if the directory cannot be read or its metadata cannot be queried.

Signature:

pub fn get_cache_metadata(cache_dir: &str) -> Result<CacheStats, Error>

Example:

let result = get_cache_metadata("value")?;

Parameters:

Name	Type	Required	Description
`cache_dir`	`&str`	Yes	The cache dir

Returns: CacheStats

Errors: Returns Err(Error).

clear_cache_directory()

Delete every file in cache_dir, returning the count and total size (MB) removed.

Returns (0, 0.0) if the directory does not exist. Files that cannot be removed are skipped with a debug log; errors do not abort the clearing operation.

Errors:

Returns an error if the directory cannot be read.

Signature:

pub fn clear_cache_directory(cache_dir: &str) -> Result<String, Error>

Example:

let result = clear_cache_directory("value")?;

Parameters:

Name	Type	Required	Description
`cache_dir`	`&str`	Yes	The cache dir

Returns: String

Errors: Returns Err(Error).

merge_config_json()

Merge extraction configuration using JSON-level field override.

Serializes the base config to JSON, merges each field from the override JSON (top-level only), and deserializes back. This correctly handles boolean fields explicitly set to their default values — the override always wins for any field present in override_json.

Fields not present in override_json are preserved from base.

Errors:

Returns Err if the base config cannot be serialized, or if the merged JSON cannot be deserialized back into ExtractionConfig (e.g., wrong field types).

Signature:

pub fn merge_config_json(base: &ExtractionConfig, override_json: &str) -> Result<ExtractionConfig, String>

Example:

use xberg::ExtractionConfig;
use serde_json::json;

let mut base = ExtractionConfig::default();
base.use_cache = true;

let overrides = r#"{"force_ocr": true}"#;
let merged = xberg::core::config::merge::merge_config_json(&base, overrides).unwrap();
assert!(merged.use_cache);   // preserved from base
assert!(merged.force_ocr);   // applied from override

Parameters:

Name	Type	Required	Description
`base`	`&ExtractionConfig`	Yes	The extraction config
`override_json`	`&str`	Yes	The override json

Returns: ExtractionConfig

Errors: Returns Err(String).

build_config_from_json()

Build extraction config by optionally merging JSON overrides into a base config.

If override_json is None, returns a clone of base after validating it. Otherwise delegates to merge_config_json, which validates the merged result.

Signature:

pub fn build_config_from_json(base: &ExtractionConfig, override_json: Option<&str>) -> Result<ExtractionConfig, String>

Example:

let result = build_config_from_json(&ExtractionConfig::default(), Some("value"))?;

Parameters:

Name	Type	Required	Description
`base`	`&ExtractionConfig`	Yes	The extraction config
`override_json`	`Option<&str>`	No	The override json

Returns: ExtractionConfig

Errors: Returns Err(String).

layout_wastes_plain_output()

Whether enabling layout detection while output_format stays Plain wastes the layout pass.

Layout detection exists to feed the structured-reconstruction pipeline that turns detected regions into headings, lists and tables. At Plain no renderer consumes that structure — the layout model still runs (measured: 20s on tesseract, 202s on paddle, for a 16-page scan) and every region it detects is discarded. This implements point 4 of the documented OCR/layout contract: “layout enabled implies output should be structured, never plain.”

This is a pure predicate meant to back a warning, not a validation error and not a coercion: Plain stays the default output format and layout stays off by default, so a caller combining both deliberately is still free to do so. Pass the fully resolved values (after every override has been applied), not raw CLI flags.

Returns:

true when layout is enabled and the output format is Plain — the layout work will be computed and its structure discarded.

Signature:

pub fn layout_wastes_plain_output(layout_enabled: bool, output_format: &crate::OutputFormat) -> bool

Example:

use xberg::core::config_validation::layout_wastes_plain_output;
use xberg::OutputFormat;

assert!(layout_wastes_plain_output(true, &OutputFormat::Plain));
assert!(!layout_wastes_plain_output(true, &OutputFormat::Markdown));
assert!(!layout_wastes_plain_output(false, &OutputFormat::Plain));

Parameters:

Name	Type	Required	Description
`layout_enabled`	`bool`	Yes	Whether layout detection will run for this extraction.
`output_format`	`&crate::OutputFormat`	Yes	The resolved output format the extraction will render.

Returns: bool

extract()

Extract content from a single bytes or URI input.

Signature:

pub async fn extract(input: ExtractInput, config: &ExtractionConfig) -> Result<ExtractionResult, Error>

Example:

let result = extract(ExtractInput::default(), &ExtractionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`input`	`ExtractInput`	Yes	The input data
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: ExtractionResult

Errors: Returns Err(Error).

extract_with_external_redaction()

Since: v1.3

Extract one bytes input and redact a JSON array or JSON Lines payload from an external inspection engine.

The payload is parsed in Rust so vendor aliases and nested fields remain intact across language bindings. offset_encoding defaults to unicode_code_points and max_findings defaults to 10,000 when omitted. Unknown encodings return a validation error.

Signature:

pub async fn extract_with_external_redaction(input: ExtractInput, config: &ExtractionConfig, findings_json: &str, offset_encoding: Option<&str>, max_findings: Option<u32>) -> Result<ExtractionResult, Error>

Example:

let result = extract_with_external_redaction(ExtractInput::default(), &ExtractionConfig::default(), "value", Some("value"), Some(42)).await?;

Parameters:

Name	Type	Required	Description
`input`	`ExtractInput`	Yes	The input data
`config`	`&ExtractionConfig`	Yes	The configuration options
`findings_json`	`&str`	Yes	The findings json
`offset_encoding`	`Option<&str>`	No	The offset encoding
`max_findings`	`Option<u32>`	No	The max findings

Returns: ExtractionResult

Errors: Returns Err(Error).

extract_batch()

Extract content from multiple bytes or URI inputs.

Signature:

pub async fn extract_batch(inputs: Vec<ExtractInput>, config: &ExtractionConfig) -> Result<ExtractionResult, Error>

Example:

let result = extract_batch(vec![], &ExtractionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`inputs`	`Vec<ExtractInput>`	Yes	The inputs
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: ExtractionResult

Errors: Returns Err(Error).

map_url()

Discover all pages and sitemaps reachable from uri without extracting document content.

Builds a crawlberg::CrawlEngine from config.crawl, calls CrawlEngine::map, and returns the set of discovered URLs as a crawlberg::MapResult (re-exported as MapResult).

Use this when you need the URL inventory of a site before committing to full document extraction — e.g. to build a crawl queue or validate scope.

Errors:

Returns Validation if the crawl configuration fails validation or if the map operation itself fails.

Signature:

pub async fn map_url(uri: &str, config: &UrlExtractionConfig) -> Result<MapResult, Error>

Example:

let result = map_url("value", &UrlExtractionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`uri`	`&str`	Yes	The uri
`config`	`&UrlExtractionConfig`	Yes	The configuration options

Returns: MapResult

Errors: Returns Err(Error).

validate_mime_type()

Validate that a MIME type is supported.

Returns:

The validated MIME type (may be normalized).

Errors:

Returns XbergError::UnsupportedFormat if not supported.

Signature:

pub fn validate_mime_type(mime_type: &str) -> Result<String, Error>

Example:

let result = validate_mime_type("value")?;

Parameters:

Name	Type	Required	Description
`mime_type`	`&str`	Yes	The MIME type to validate

Returns: String

Errors: Returns Err(Error).

list_supported_formats()

List all supported document formats.

Returns every file extension Xberg recognizes together with its corresponding MIME type, derived from the central format registry. Formats that have no registered file extension (such as source code, which is detected dynamically) are not included.

The static EXT_TO_MIME table lists every format the codebase knows how to describe, regardless of which Cargo features were compiled in. Advertising that table directly would claim support for extractors that may not exist in this build (see GH#1387). To keep the advertised catalogue honest, the table is intersected with the document extractor registry: an extension is only included if some registered extractor actually claims its MIME type in this build. This can never drift from reality and automatically covers third-party extractors registered at runtime.

The list is sorted alphabetically by file extension.

Returns:

A vector of SupportedFormat entries sorted by extension, limited to formats with a registered extractor in this build.

Signature:

pub fn list_supported_formats() -> Vec<SupportedFormat>

Example:

use xberg::core::mime::list_supported_formats;

let formats = list_supported_formats();
assert!(!formats.is_empty());

Returns: Vec<SupportedFormat>

clear_processor_cache()

Clear the processor cache (primarily for testing when registry changes).

Signature:

pub fn clear_processor_cache() -> Result<(), Error>

Example:

clear_processor_cache()?;

Returns: No return value.

Errors: Returns Err(Error).

apply_output_format()

Apply output format conversion to the extraction result.

Records the output format in metadata and swaps in pre-rendered content (produced during derive_extraction_result) if available.

This runs as the final pipeline step, after post-processors have operated on the plain-text content field.

Signature:

pub fn apply_output_format(result: ExtractedDocument, output_format: OutputFormat) -> ExtractedDocument

Example:

let result = apply_output_format(ExtractedDocument::default(), OutputFormat::Plain);

Parameters:

Name	Type	Required	Description
`result`	`ExtractedDocument`	Yes	The extraction result to modify
`output_format`	`OutputFormat`	Yes	The desired output format

Returns: ExtractedDocument

run_pipeline()

Run the post-processing pipeline on an InternalDocument.

Derives ExtractedDocument from InternalDocument via the derivation pipeline, then executes post-processing in the following order:

Post-Processors - Execute by stage (Early immediately; Middle/Late after a provisional chunking pass, so chunk-aware post-processors see result.chunks)
Language detection
Quality Processing - Token reduction, NFC normalization, output-format application
Validators - Run validation hooks on the processed result (can fail fast)
Chunking (final) - Text splitting if enabled
Encoded output - A binary format such as DOCX packages the finished rendering, leaving base64 in content; its chunks describe the Markdown it was built from

Chunking runs twice when Middle/Late post-processors are active: once right after Early post-processors (so a chunk-aware post-processor sees non-empty result.chunks, per the tested contract in test_middle_postprocessors_run_after_explicit_chunking), and again as the last step, after every step that rewrites content (token reduction, NFC normalization, output-format swapping). Only the final pass’s result.chunks is returned. Chunk byte_start/byte_end (and page-derived first_page/last_page) are byte offsets into whatever content looked like at chunking time, and are the join key consumers use for highlighting and rehydration — running chunking only once, early, left those offsets silently indexing stale content once later steps mutated it (#213). The first pass’s offsets are provisional; a Middle/Late post-processor must not treat them as final. This trades an extra (usually cheap) chunking pass — and, if embeddings are configured, a discarded first embedding batch — for offset correctness in the returned result.

Returns:

The processed extraction result.

Errors:

Validator errors bubble up immediately
Post-processor errors are caught and recorded in metadata
System errors (IO, RuntimeError equivalents) always bubble up

Signature:

pub async fn run_pipeline(doc: InternalDocument, config: &ExtractionConfig) -> Result<ExtractedDocument, Error>

Example:

let result = run_pipeline(InternalDocument::default(), &ExtractionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`doc`	`InternalDocument`	Yes	The internal document produced by the extractor
`config`	`&ExtractionConfig`	Yes	Extraction configuration

Returns: ExtractedDocument

Errors: Returns Err(Error).

derive_extraction_result()

Derive a complete ExtractedDocument from an InternalDocument.

This is the main entry point for the derivation pipeline. It:

Resolves relationships (needed by renderers for footnotes)
Renders plain-text content (for post-processors)
Pre-renders formatted content if output_format != Plain
Groups elements by page into PageContent
Extracts OCR elements for backward compatibility
Optionally derives DocumentStructure (assumes relationships resolved)
Assembles the final ExtractedDocument

Signature:

pub fn derive_extraction_result(doc: InternalDocument, include_document_structure: bool, output_format: crate::core::config::OutputFormat) -> ExtractedDocument

Example:

let result = derive_extraction_result(InternalDocument::default(), true, crate::core::config::OutputFormat::Plain);

Parameters:

Name	Type	Required	Description
`doc`	`InternalDocument`	Yes	The internal document
`include_document_structure`	`bool`	Yes	The include document structure
`output_format`	`crate::core::config::OutputFormat`	Yes	The output format

Returns: ExtractedDocument

classify()

Classify an image based on its metadata and visual properties.

Uses a rule cascade over already-captured signals: dimensions, aspect ratio, colorspace, bits-per-component, format, and histogram entropy on a downsampled 64×64 thumbnail.

Returns:

A tuple of (ImageKind, confidence) where confidence is in [0.0, 1.0]. Returns (Unknown, 0.0) if bytes cannot be decoded.

Signature:

pub fn classify(input: ImageClassifyInput) -> String

Example:

let result = classify(todo!());

Parameters:

Name	Type	Required	Description
`input`	`ImageClassifyInput`	Yes	The input data

Returns: String

cluster_tiles()

Cluster spatially adjacent, similarly-sized images on a page.

Groups images that appear to be tiles of a single figure (e.g., a technical drawing composed of dozens of raster fragments). For each group with 2+ members, assigns a shared cluster_id and reclassifies members as TileFragment.

Clustering criteria:

Images must be on the same page
Images must be classified as Drawing, Diagram, or TileFragment (or unclassified with area < 300×300)
Bounding boxes (if present) must be spatially adjacent: within half a tile-side (min(width, height) / 2) of each other
Dimensions must match within ±20%
Emits one info! span per page with cluster count and max cluster size

Signature:

pub fn cluster_tiles(images: &mut [ExtractedImage])

Example:

cluster_tiles(&mut vec![]);

Parameters:

Name	Type	Required	Description
`images`	`&mut \[ExtractedImage\]`	Yes	The images

Returns: No return value.

convert_internal_elements_to_elements()

Walk an InternalDocument in document reading order and convert to Elements.

This preserves the extractor’s native element order, which is critical for formats like DOCX that have no native page boundaries: per-page reconstruction reorders elements by page, but the flat InternalDocument.elements list is always in reading order as the extractor encountered the content.

Container markers (ListStart, ListEnd, QuoteStart, QuoteEnd, GroupStart, GroupEnd) are structural bookkeeping and are skipped — they carry no text content.

Returns:

A vector of Elements in the extractor’s native reading order.

Signature:

pub fn convert_internal_elements_to_elements(doc: &InternalDocument, filename: Option<&str>) -> Vec<Element>

Example:

let result = convert_internal_elements_to_elements(&InternalDocument::default(), Some("value"));

Parameters:

Name	Type	Required	Description
`doc`	`&InternalDocument`	Yes	The `InternalDocument` from the extractor, before per-page reconstruction
`filename`	`Option<&str>`	No	Document title for element metadata, forwarded from `result.metadata.title`

Returns: Vec<Element>

transform_extraction_result_to_elements()

Transform an extraction result into semantic elements.

This function takes a reference to an ExtractedDocument and generates a vector of Element structs representing semantic blocks in the document. It detects content sections, list items, page breaks, and other structural elements to create an Unstructured-compatible element-based output.

Handles:

PDF hierarchy → Title/Heading elements
Multi-page documents with correct page numbers
Table and Image extraction
PageBreak interleaving
Bounding box coordinates
Paragraph detection for NarrativeText

When result.internal_document is Some, walks it directly in document reading order instead of reassembling from result.pages. This preserves DOCX element order, which is otherwise scrambled by per-page reconstruction.

Returns:

A vector of Elements with proper semantic types and metadata.

Signature:

pub fn transform_extraction_result_to_elements(result: &ExtractedDocument) -> Vec<Element>

Example:

let result = transform_extraction_result_to_elements(&ExtractedDocument::default());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	Reference to the ExtractedDocument to transform

Returns: Vec<Element>

excel_to_text()

Convert an Excel workbook to plain text (space-separated cells, one row per line).

Each sheet is separated by a blank line. Sheet names are included as headers. This produces text suitable for quality scoring against ground truth.

Signature:

pub fn excel_to_text(workbook: &ExcelWorkbook) -> String

Example:

let result = excel_to_text(&todo!());

Parameters:

Name	Type	Required	Description
`workbook`	`&ExcelWorkbook`	Yes	The excel workbook

Returns: String

excel_to_markdown()

Render all sheets in an ExcelWorkbook as a single Markdown string.

Sheets are separated by double newlines; each sheet’s pre-rendered markdown is trimmed of trailing whitespace before joining.

Signature:

pub fn excel_to_markdown(workbook: &ExcelWorkbook) -> String

Example:

let result = excel_to_markdown(&todo!());

Parameters:

Name	Type	Required	Description
`workbook`	`&ExcelWorkbook`	Yes	The excel workbook

Returns: String

convert_html_to_markdown()

Convert HTML with optional configuration and output format.

Uses sensible defaults if no configuration is provided:

extract_metadata = true (parse YAML frontmatter)
include_document_structure = true (populate document tree)
preprocessing.enabled = true (clean navigation and boilerplate before conversion)

Supports both markdown and djot output based on the output_format parameter. Defaults to Markdown for backward compatibility.

WASM Limitations

In WASM builds, HTML files larger than 2MB will be rejected with an error to prevent stack overflow. For larger files, use the native library.

Returns:

A markdown or djot string, or an error if conversion fails

Signature:

pub fn convert_html_to_markdown(html: &str, options: Option<ConversionOptions>, output_format: Option<XbergOutputFormat>) -> Result<String, XbergError>

Example:

let result = convert_html_to_markdown("value", Some(ConversionOptions::default()), Some(todo!()))?;

Parameters:

Name	Type	Required	Description
`html`	`&str`	Yes	The HTML string to convert
`options`	`Option<ConversionOptions>`	No	Optional conversion options; defaults will be used if None
`output_format`	`Option<XbergOutputFormat>`	No	Optional output format; defaults to Markdown if None

Returns: String

Errors: Returns Err(XbergError).

decode_doc_security_flags()

Decode a DocSecurity bit field into named boolean flags.

DocSecurity (ECMA-376 Part 1 §22.2.2.7, as clarified by MS-OI29500) packs four independent restrictions into one integer: 1 = password protected, 2 = read-only recommended, 4 = read-only enforced, 8 = locked for annotation. doc_security (see DocxAppProperties::doc_security and its XLSX/PPTX equivalents) is parsed and stored as that raw integer; extractors decode it here so consumers see named flags in Metadata::additional rather than an opaque bit field (#230).

The pairs are returned in ascending bit order. Higher-order bits beyond 8 are not part of the schema and are ignored.

All four flags are always returned, including when raw is 0: an explicit false records that the document declares no security restrictions, as opposed to the doc_security: None case (no DocSecurity element at all), where nothing should be decoded because there is no data to decode. Security-relevant booleans are exactly the kind of value where “absent” and “false” must not be conflated by convention.

Returns (key, value) pairs; the keys are stable strings suitable for use in a Metadata::additional map.

Signature:

pub fn decode_doc_security_flags(raw: i32) -> String

Example:

let result = decode_doc_security_flags(42);

Parameters:

Name	Type	Required	Description
`raw`	`i32`	Yes	The raw

Returns: String

djot_content_to_djot()

Convert DjotContent back to djot markup.

This function takes a DjotContent structure and generates valid djot markup from it, preserving:

Block structure (headings, code blocks, lists, blockquotes, etc.)
Inline formatting (strong, emphasis, highlight, subscript, superscript, etc.)
Attributes where present ({.class #id key=“value”})

Returns:

A String containing valid djot markup

Signature:

pub fn djot_content_to_djot(content: &crate::types::DjotContent) -> String

Example:

let djot_content = // ... extract from some source
let markup = djot_content_to_djot(&djot_content);
println!("{}", markup);

Parameters:

Name	Type	Required	Description
`content`	`&crate::types::DjotContent`	Yes	The DjotContent to convert

Returns: String

extraction_result_to_djot()

Convert any ExtractedDocument to djot format.

This function converts an ExtractedDocument to djot markup:

If djot_content is Some, uses djot_content_to_djot for full fidelity conversion
Otherwise, wraps the plain text content in paragraphs

Returns:

A Result containing the djot markup string

Signature:

pub fn extraction_result_to_djot(result: &crate::types::ExtractedDocument) -> Result<String, XbergError>

Example:

let result = extractor.extract_bytes(bytes, "text/plain", &config).await?;
let djot_markup = extraction_result_to_djot(&result)?;

Parameters:

Name	Type	Required	Description
`result`	`&crate::types::ExtractedDocument`	Yes	The ExtractedDocument to convert

Returns: String

Errors: Returns Err(XbergError).

normalize_whitespace()

Normalize whitespace in a string.

Collapses multiple consecutive spaces/tabs into a single space
Preserves single newlines (paragraph breaks from \par)
Collapses multiple consecutive newlines into a double newline
Trims leading/trailing whitespace from each line
Trims leading/trailing blank lines

Signature:

pub fn normalize_whitespace(s: &str) -> String

Example:

let result = normalize_whitespace("value");

Parameters:

Name	Type	Required	Description
`s`	`&str`	Yes	The s

Returns: String

register_embedding_backend()

The backend will be keyed by its Plugin::name() and can be referenced from Plugin by the same name.

Errors:

Validation if the name is empty, contains whitespace, or dimensions() is zero.
Plugin if a backend with that name is already registered.
Any error from the backend’s initialize() method.

Signature:

pub fn register_embedding_backend(backend: EmbeddingBackend) -> Result<(), XbergError>

Example:

register_embedding_backend(todo!())?;

Parameters:

Name	Type	Required	Description
`backend`	`EmbeddingBackend`	Yes	The embedding backend

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_embedding_backend()

Unregister an embedding backend by name, calling its shutdown() method.

No-op if the backend is not registered.

Errors:

Any error returned by the backend’s shutdown() method.

Signature:

pub fn unregister_embedding_backend(name: &str) -> Result<(), XbergError>

Example:

unregister_embedding_backend("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

clear_embedding_backends()

Clear all embedding backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

pub fn clear_embedding_backends() -> Result<(), XbergError>

Example:

clear_embedding_backends()?;

Returns: No return value.

Errors: Returns Err(XbergError).

list_embedding_backends()

List the names of all registered embedding backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

pub fn list_embedding_backends() -> Result<Vec<String>, XbergError>

Example:

let result = list_embedding_backends()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

register_document_extractor()

The extractor is keyed by name and indexed for every MIME type returned by supported_mime_types.

Errors:

Validation if the plugin name is empty or contains whitespace.
Any error returned by the extractor’s initialize() method.

Signature:

pub fn register_document_extractor(extractor: DocumentExtractor) -> Result<(), XbergError>

Example:

register_document_extractor(todo!())?;

Parameters:

Name	Type	Required	Description
`extractor`	`DocumentExtractor`	Yes	The document extractor

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_document_extractor()

Unregister a document extractor by name.

Removes the extractor from the global registry and calls its shutdown() method. No-op if no extractor with that name is registered.

Errors:

Any error returned by the extractor’s shutdown() method.

Signature:

pub fn unregister_document_extractor(name: &str) -> Result<(), XbergError>

Example:

unregister_document_extractor("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

list_document_extractors()

List names of all registered document extractors.

Signature:

pub fn list_document_extractors() -> Result<Vec<String>, XbergError>

Example:

let result = list_document_extractors()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

clear_document_extractors()

Clear all document extractors from the global registry.

Calls shutdown() on every registered extractor, then empties the registry.

Errors:

Any error returned by an extractor’s shutdown() method. The first error encountered stops processing of remaining extractors.

Signature:

pub fn clear_document_extractors() -> Result<(), XbergError>

Example:

clear_document_extractors()?;

Returns: No return value.

Errors: Returns Err(XbergError).

register_ocr_backend()

The OCR backend will be registered with its name from the name() method and can be used for OCR processing via the extraction pipeline.

Returns:

Ok(()) if registration succeeded
Err(...) if validation failed or initialization failed

Errors:

XbergError::Validation - Invalid backend name (empty or contains whitespace)
Any error from the backend’s initialize() method

Signature:

pub fn register_ocr_backend(backend: OcrBackend) -> Result<(), XbergError>

Example:

use xberg::plugins::{Plugin, OcrBackend, register_ocr_backend, OcrBackendType};
use xberg::{Result, OcrConfig};
use xberg::types::{ExtractedDocument, Metadata};
use async_trait::async_trait;
use std::borrow::Cow;
use std::sync::Arc;
use std::path::Path;

struct CustomOcr;

impl Plugin for CustomOcr {
    fn name(&self) -> &str { "custom-ocr" }
    fn version(&self) -> String { "1.0.0".to_string() }
    fn initialize(&self) -> Result<()> { Ok(()) }
    fn shutdown(&self) -> Result<()> { Ok(()) }
}

#[async_trait]
impl OcrBackend for CustomOcr {
    async fn process_image(&self, _: &[u8], _: &OcrConfig) -> Result<ExtractedDocument> {
        let mut document = ExtractedDocument::default();
        document.content = "text".to_string();
        document.mime_type = Cow::Borrowed("text/plain");
        Ok(document)
    }
    fn supports_language(&self, _: &str) -> bool { true }
    fn backend_type(&self) -> OcrBackendType { OcrBackendType::Custom }
}

let backend = Arc::new(CustomOcr);
register_ocr_backend(backend)?;

Parameters:

Name	Type	Required	Description
`backend`	`OcrBackend`	Yes	The OCR backend implementation wrapped in Arc

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_ocr_backend()

Unregister an OCR backend by name.

Removes the OCR backend from the global registry and calls its shutdown() method.

Returns:

Ok(()) if the backend was unregistered or didn’t exist
Err(...) if the shutdown method failed

Signature:

pub fn unregister_ocr_backend(name: &str) -> Result<(), XbergError>

Example:

use xberg::plugins::unregister_ocr_backend;

unregister_ocr_backend("custom-ocr")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	Name of the OCR backend to unregister

Returns: No return value.

Errors: Returns Err(XbergError).

list_ocr_backends()

List all registered OCR backends.

Returns the names of all OCR backends currently registered in the global registry.

Returns:

A vector of OCR backend names.

Signature:

pub fn list_ocr_backends() -> Result<Vec<String>, XbergError>

Example:

use xberg::plugins::list_ocr_backends;

let backends = list_ocr_backends()?;
for name in backends {
    println!("Registered OCR backend: {}", name);
}

Returns: Vec<String>

Errors: Returns Err(XbergError).

list_ocr_backend_capabilities()

List every registered OCR backend’s name alongside its declared supported languages.

This is the capability-enumeration counterpart to list_ocr_backends: where that function exposes only backend names, this exposes each backend’s supported_languages() too, so a consumer (for example, a job-acceptance gate) does not need to hardcode a second list of backend languages.

Determinism

The returned vector is sorted by name, regardless of registration order or the order reported by the underlying registry. supported_languages within each entry is not sorted — see OcrBackendCapabilities::supported_languages for why.

Cost

Calling this is not free for every backend. In particular, TesseractBackend’s supported_languages() allocates a Tesseract API and initializes it against the same tessdata directory a real OCR job resolves (resolve_tessdata_path), the first time it is called, to enumerate installed tessdata languages; subsequent calls are served from a cache.

Errors:

Returns an error only if the registry lock cannot be acquired in the current environment.

Signature:

pub fn list_ocr_backend_capabilities() -> Result<Vec<OcrBackendCapabilities>, XbergError>

Example:

use xberg::plugins::list_ocr_backend_capabilities;

for capability in list_ocr_backend_capabilities()? {
    println!("{}: {:?}", capability.name, capability.supported_languages);
}

Returns: Vec<OcrBackendCapabilities>

Errors: Returns Err(XbergError).

list_ocr_backend_capabilities_for()

list_ocr_backend_capabilities, reporting each backend’s languages under config.

Tesseract’s language list is a property of the tessdata directory it resolves, and config.tessdata_path is the first entry of that search chain. The config-less form always answers for the no-override chain, which can be a different directory than the one a job using config will load from. Use this form when config.tessdata_path is set. See GH#1857.

Signature:

pub fn list_ocr_backend_capabilities_for(config: &OcrConfig) -> Result<Vec<OcrBackendCapabilities>, XbergError>

Example:

let result = list_ocr_backend_capabilities_for(&OcrConfig::default())?;

Parameters:

Name	Type	Required	Description
`config`	`&OcrConfig`	Yes	The configuration options

Returns: Vec<OcrBackendCapabilities>

Errors: Returns Err(XbergError).

ocr_backend_supports_language()

Check whether a specific registered OCR backend supports a language.

Delegates to the named backend’s own OcrBackend::supports_language, which is the correct per-language decision — do not infer support (or its absence) from whether list_ocr_backend_capabilities reports an empty supported_languages list for that backend, since an empty list can mean “does not enumerate” rather than “supports nothing” (see OcrBackendCapabilities::supported_languages).

Errors:

Returns an error if no backend with that name (or alias) is registered.

Signature:

pub fn ocr_backend_supports_language(backend: &str, language: &str) -> Result<bool, XbergError>

Example:

use xberg::plugins::ocr_backend_supports_language;

let supported = ocr_backend_supports_language("tesseract", "eng")?;

Parameters:

Name	Type	Required	Description
`backend`	`&str`	Yes	Name of a registered OCR backend, as returned by `list_ocr_backends`. Lookup is case-insensitive and resolves the same `paddleocr` alias as backend dispatch.
`language`	`&str`	Yes	Language code to check (e.g. `"eng"`, `"deu"`).

Returns: bool

Errors: Returns Err(XbergError).

ocr_backend_supports_language_for()

ocr_backend_supports_language, answering under config.

Use this, not the config-less form, when the caller sets OcrConfig.tessdata_path: the config-less form checks the no-override search chain and can deny a language the job would load without trouble. See GH#1857.

Signature:

pub fn ocr_backend_supports_language_for(backend: &str, language: &str, config: &OcrConfig) -> Result<bool, XbergError>

Example:

let result = ocr_backend_supports_language_for("value", "value", &OcrConfig::default())?;

Parameters:

Name	Type	Required	Description
`backend`	`&str`	Yes	The backend
`language`	`&str`	Yes	The language
`config`	`&OcrConfig`	Yes	The configuration options

Returns: bool

Errors: Returns Err(XbergError).

clear_ocr_backends()

Clear all OCR backends from the global registry.

Removes all OCR backends and calls their shutdown() methods.

Returns:

Ok(()) if all backends were cleared successfully
Err(...) if any shutdown method failed

Signature:

pub fn clear_ocr_backends() -> Result<(), XbergError>

Example:

use xberg::plugins::clear_ocr_backends;

clear_ocr_backends()?;

Returns: No return value.

Errors: Returns Err(XbergError).

register()

Signature:

pub fn register() -> Result<(), XbergError>

Example:

register()?;

Returns: No return value.

Errors: Returns Err(XbergError).

list_post_processors()

List all registered post-processor names.

Returns a vector of all post-processor names currently registered in the global registry.

Returns:

Ok(Vec<String>) - Vector of post-processor names
Err(...) if the registry lock is poisoned

Signature:

pub fn list_post_processors() -> Result<Vec<String>, XbergError>

Example:

use xberg::plugins::list_post_processors;

let processors = list_post_processors()?;
for name in processors {
    println!("Registered post-processor: {}", name);
}

Returns: Vec<String>

Errors: Returns Err(XbergError).

register_post_processor()

The processor’s execution priority is taken from its PostProcessor::priority method (default: 50). Higher values run first within the same processing stage.

Errors:

Returns a retryable in-use error when an extraction is executing a processor snapshot.

Signature:

pub fn register_post_processor(processor: PostProcessor) -> Result<(), XbergError>

Example:

register_post_processor(todo!())?;

Parameters:

Name	Type	Required	Description
`processor`	`PostProcessor`	Yes	The post-processor to register

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_post_processor()

Unregister a post-processor by name.

Automatic built-in recovery keeps this name suppressed until it is explicitly registered again or clear_post_processors resets the complete registry. Returns a retryable in-use error when an extraction is executing a processor snapshot.

Signature:

pub fn unregister_post_processor(name: &str) -> Result<(), XbergError>

Example:

unregister_post_processor("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

clear_post_processors()

Remove all registered post-processors.

The next post-processed extraction restores enabled built-in processors before it snapshots the processor cache. Custom processors remain removed. Use unregister_post_processor when one named processor should remain absent while the rest of the registry stays intact. Returns a retryable in-use error when an extraction is executing a processor snapshot.

Signature:

pub fn clear_post_processors() -> Result<(), XbergError>

Example:

clear_post_processors()?;

Returns: No return value.

Errors: Returns Err(XbergError).

get_ocr_backend_registry()

Get the global OCR backend registry.

Signature:

pub fn get_ocr_backend_registry() -> OcrBackendRegistry

Example:

let result = get_ocr_backend_registry();

Returns: OcrBackendRegistry

get_embedding_backend_registry()

Get the global embedding backend registry.

Signature:

pub fn get_embedding_backend_registry() -> EmbeddingBackendRegistry

Example:

let result = get_embedding_backend_registry();

Returns: EmbeddingBackendRegistry

get_reranker_backend_registry()

Get the global reranker backend registry.

Signature:

pub fn get_reranker_backend_registry() -> RerankerBackendRegistry

Example:

let result = get_reranker_backend_registry();

Returns: RerankerBackendRegistry

get_tokenizer_backend_registry()

Get the global tokenizer backend registry.

Signature:

pub fn get_tokenizer_backend_registry() -> TokenizerBackendRegistry

Example:

let result = get_tokenizer_backend_registry();

Returns: TokenizerBackendRegistry

get_document_extractor_registry()

Get the global document extractor registry.

Signature:

pub fn get_document_extractor_registry() -> DocumentExtractorRegistry

Example:

let result = get_document_extractor_registry();

Returns: DocumentExtractorRegistry

get_post_processor_registry()

Get the global post-processor registry.

Signature:

pub fn get_post_processor_registry() -> PostProcessorRegistry

Example:

let result = get_post_processor_registry();

Returns: PostProcessorRegistry

get_validator_registry()

Get the global validator registry.

Signature:

pub fn get_validator_registry() -> ValidatorRegistry

Example:

let result = get_validator_registry();

Returns: ValidatorRegistry

get_renderer_registry()

Get the global renderer registry.

Signature:

pub fn get_renderer_registry() -> RendererRegistry

Example:

let result = get_renderer_registry();

Returns: RendererRegistry

register_renderer()

The renderer’s format name is taken from Plugin::name. Registering a renderer with a name that already exists replaces the previous renderer for that format.

Note on `Result` return type

Returns Result<()> for cross-language API symmetry required by the alef trait-bridge codegen. The underlying parking_lot::RwLock cannot be poisoned (parking_lot provides no poisoning semantics), so this function never returns Err in practice.

Signature:

pub fn register_renderer(renderer: Renderer) -> Result<(), XbergError>

Example:

register_renderer(todo!())?;

Parameters:

Name	Type	Required	Description
`renderer`	`Renderer`	Yes	The renderer

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_renderer()

Unregister a renderer by format name.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

pub fn unregister_renderer(name: &str) -> Result<(), XbergError>

Example:

unregister_renderer("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

list_renderers()

List names of all registered renderers.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

pub fn list_renderers() -> Result<Vec<String>, XbergError>

Example:

let result = list_renderers()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

clear_renderers()

Clear all renderers from the global registry.

Removes every renderer, including the built-in defaults (markdown, html, djot, plain). After calling this no renderers are registered; re-register as needed.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

pub fn clear_renderers() -> Result<(), XbergError>

Example:

clear_renderers()?;

Returns: No return value.

Errors: Returns Err(XbergError).

register_reranker_backend()

The backend will be keyed by its Plugin::name() and can be referenced from Plugin by the same name.

Errors:

Validation if the name is empty or contains whitespace.
Plugin if a backend with that name is already registered.
Any error from the backend’s initialize() method.

Signature:

pub fn register_reranker_backend(backend: RerankerBackend) -> Result<(), XbergError>

Example:

register_reranker_backend(todo!())?;

Parameters:

Name	Type	Required	Description
`backend`	`RerankerBackend`	Yes	The reranker backend

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_reranker_backend()

Unregister a reranker backend by name, calling its shutdown() method.

No-op if the backend is not registered.

Errors:

Any error returned by the backend’s shutdown() method.

Signature:

pub fn unregister_reranker_backend(name: &str) -> Result<(), XbergError>

Example:

unregister_reranker_backend("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

clear_reranker_backends()

Clear all reranker backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

pub fn clear_reranker_backends() -> Result<(), XbergError>

Example:

clear_reranker_backends()?;

Returns: No return value.

Errors: Returns Err(XbergError).

list_reranker_backends()

List the names of all registered reranker backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

pub fn list_reranker_backends() -> Result<Vec<String>, XbergError>

Example:

let result = list_reranker_backends()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

register_tokenizer_backend()

The backend is keyed by its Plugin::name(). Token-budgeted chunking resolves ChunkSizing::Tokenizer { model } against this registry first — a registered name takes precedence over a HuggingFace model id — and falls back to the HuggingFace path when the name is not registered.

Errors:

Validation if the name is empty, contains whitespace, or the backend reports zero tokens for non-empty text.
Plugin if a backend with that name is already registered.
Any error from the backend’s initialize() method.

Signature:

pub fn register_tokenizer_backend(backend: TokenizerBackend) -> Result<(), XbergError>

Example:

register_tokenizer_backend(todo!())?;

Parameters:

Name	Type	Required	Description
`backend`	`TokenizerBackend`	Yes	The tokenizer backend

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_tokenizer_backend()

Unregister a tokenizer backend by name, calling its shutdown() method.

No-op if the backend is not registered.

Errors:

Any error returned by the backend’s shutdown() method.

Signature:

pub fn unregister_tokenizer_backend(name: &str) -> Result<(), XbergError>

Example:

unregister_tokenizer_backend("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

clear_tokenizer_backends()

Clear all tokenizer backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

pub fn clear_tokenizer_backends() -> Result<(), XbergError>

Example:

clear_tokenizer_backends()?;

Returns: No return value.

Errors: Returns Err(XbergError).

list_tokenizer_backends()

List the names of all registered tokenizer backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

pub fn list_tokenizer_backends() -> Result<Vec<String>, XbergError>

Example:

let result = list_tokenizer_backends()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

register_validator()

Signature:

pub fn register_validator(validator: Validator) -> Result<(), XbergError>

Example:

register_validator(todo!())?;

Parameters:

Name	Type	Required	Description
`validator`	`Validator`	Yes	The validator

Returns: No return value.

Errors: Returns Err(XbergError).

unregister_validator()

Unregister a validator by name.

Signature:

pub fn unregister_validator(name: &str) -> Result<(), XbergError>

Example:

unregister_validator("value")?;

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: No return value.

Errors: Returns Err(XbergError).

list_validators()

List names of all registered validators.

Signature:

pub fn list_validators() -> Result<Vec<String>, XbergError>

Example:

let result = list_validators()?;

Returns: Vec<String>

Errors: Returns Err(XbergError).

clear_validators()

Remove all registered validators.

Signature:

pub fn clear_validators() -> Result<(), XbergError>

Example:

clear_validators()?;

Returns: No return value.

Errors: Returns Err(XbergError).

render_json()

Render an InternalDocument as a JSON tree string.

Walks the flat element list and builds a heading-driven section tree. Returns a JSON string (always valid JSON).

Signature:

pub fn render_json(doc: &InternalDocument) -> String

Example:

let result = render_json(&InternalDocument::default());

Parameters:

Name	Type	Required	Description
`doc`	`&InternalDocument`	Yes	The internal document

Returns: String

calculate_quality_score()

Score the cleanliness and readability of extracted text on [0.0, 1.0].

1.0 is the neutral score for clean prose; penalties (OCR artifacts, embedded script/style noise, navigation chrome) subtract, structural cues (headings, punctuation) add. The result is clamped to [0.0, 1.0]. It measures only the text supplied to this function and therefore cannot measure extraction completeness or recall. Callers must inspect extraction warnings separately for known omissions or degraded processing.

Pass metadata as None when the caller has no extraction metadata available; the metadata bonus simply isn’t applied in that case. Texts shorter than MIN_TEXT_LENGTH short-circuit to 0.1 regardless of metadata.

Signature:

pub fn calculate_quality_score(text: &str, metadata: Option<&HashMap<String, serde_json::Value>>) -> f64

Example:

let result = calculate_quality_score("value", Some(&std::collections::HashMap::new()));

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text
`metadata`	`Option<&HashMap<String, serde_json::Value>>`	No	The metadata

Returns: f64

classify_chunks()

Run chunk classification against an extraction result.

Mutates ChunkMetadata::classifications on every chunk in result.chunks and appends every LLM call’s usage to result.llm_usage. A chunk whose classification batch call fails (or that the model omitted from its response) is simply left with an empty classifications vector for that chunk, unless the failure is a validation error (empty config) or every batch task fails, in which case the first error is returned.

Errors:

Returns Validation when config.definitions is empty. Returns the first batch error encountered when rendering the prompt or calling the LLM fails for every batch; partial failures on a subset of batches are recorded here as a ProcessingWarning on result instead of aborting the whole run.

Signature:

pub async fn classify_chunks(result: &mut ExtractedDocument, config: &ChunkClassificationConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = classify_chunks(&mut ExtractedDocument::default(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ChunkClassificationConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

classify_chunks_owned()

Classify a document’s chunks and return the updated document.

This owned form preserves the mutations when the document crosses a language-binding boundary. Rust callers that already own a mutable document can use classify_chunks to avoid moving it.

Errors:

Returns the same validation and LLM errors as classify_chunks.

Signature:

pub async fn classify_chunks_owned(result: ExtractedDocument, config: &ChunkClassificationConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = classify_chunks_owned(ExtractedDocument::default(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`ExtractedDocument`	Yes	The extracted document
`config`	`&ChunkClassificationConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

manifest()

Returns the GLiNER files expected by xberg cache manifest.

Signature:

pub fn manifest() -> Vec<GlinerManifestEntry>

Example:

let result = manifest();

Returns: Vec<GlinerManifestEntry>

get_or_init_backend()

Return the process-wide cached GLiNER backend for the requested model.

Model download and ONNX initialization run on Tokio’s blocking pool. Concurrent callers requesting the same canonical model and runtime thread budget receive clones of the same Arc. Initialization errors are not cached, so a later call can retry after a transient download or filesystem failure. Warm cache hits return immediately without scheduling work on the blocking pool.

Errors:

Returns an error when the model name is invalid, its artifacts cannot be loaded, ONNX initialization fails, or the blocking initialization task cannot complete.

Signature:

pub async fn get_or_init_backend(model_name: Option<&str>) -> Result<String, XbergError>

Example:

let result = get_or_init_backend(Some("value")).await?;

Parameters:

Name	Type	Required	Description
`model_name`	`Option<&str>`	No	The model name

Returns: String

Errors: Returns Err(XbergError).

split_into_windows()

Split text into overlapping windows of at most window_tokens word tokens.

Returns an empty vector when text holds no word tokens at all. Every window starts and ends on a token boundary, so no mention is ever cut in half by the split itself; overlap_tokens then guarantees that a mention near a boundary is still contained whole in one of the two windows.

Signature:

pub fn split_into_windows(text: &str, window_tokens: usize, overlap_tokens: usize) -> Vec<TextWindow>

Example:

let result = split_into_windows("value", 42, 42);

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text
`window_tokens`	`usize`	Yes	The window tokens
`overlap_tokens`	`usize`	Yes	The overlap tokens

Returns: Vec<TextWindow>

merge_windowed_entities()

Fold per-window detections back into a single source-coordinate stream.

window_offsets and per_window are parallel: per_window[i] holds the entities detected in the window starting at window_offsets[i]. Entities are shifted into source coordinates, sorted, and deduplicated — the overlap region between adjacent windows otherwise reports the same mention twice. Where two same-category spans overlap the longer one wins; spans that cannot be expressed as u32 source offsets are dropped rather than reported wrong.

Signature:

pub fn merge_windowed_entities(window_offsets: &[usize], per_window: Vec<Vec<Entity>>) -> Vec<Entity>

Example:

let result = merge_windowed_entities(&vec![], vec![]);

Parameters:

Name	Type	Required	Description
`window_offsets`	`&\[usize\]`	Yes	The window offsets
`per_window`	`Vec<Vec<Entity>>`	Yes	The per window

Returns: Vec<Entity>

entities_for_every_occurrence()

Resolve mention into one Entity per occurrence in text.

Backends that report a mention without offsets must not stop at the first hit: redacting only the first occurrence leaves every later copy of the same PII in the output (xberg-io/xberg#200). Occurrences are non-overlapping and returned in ascending byte order; an empty mention yields nothing.

Signature:

pub fn entities_for_every_occurrence(text: &str, mention: &str, category: EntityCategory, confidence: Option<f32>) -> Vec<Entity>

Example:

let result = entities_for_every_occurrence("value", "value", EntityCategory::Person, Some(0.5));

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text
`mention`	`&str`	Yes	The mention
`category`	`EntityCategory`	Yes	The entity category
`confidence`	`Option<f32>`	No	The confidence

Returns: Vec<Entity>

manifest()

Expected GLiNER cache artifacts for manifest tooling.

Signature:

pub fn manifest() -> Vec<GlinerManifestEntry>

Example:

let result = manifest();

Returns: Vec<GlinerManifestEntry>

redact_external()

Since: v1.3

Redact an owned document using a JSON array or JSON Lines payload from an external inspection engine.

Signature:

pub async fn redact_external(document: ExtractedDocument, config: RedactionConfig, findings_json: &str, offset_encoding: Option<&str>, max_findings: Option<u32>) -> Result<ExtractedDocument, XbergError>

Example:

let result = redact_external(ExtractedDocument::default(), RedactionConfig::default(), "value", Some("value"), Some(42)).await?;

Parameters:

Name	Type	Required	Description
`document`	`ExtractedDocument`	Yes	The extracted document
`config`	`RedactionConfig`	Yes	The configuration options
`findings_json`	`&str`	Yes	The findings json
`offset_encoding`	`Option<&str>`	No	The offset encoding
`max_findings`	`Option<u32>`	No	The max findings

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

redact_capturing_rehydration_map()

Like redact, additionally returning the token to original-text map for later rehydration. Only RedactionStrategy::TokenReplace allocations appear in the map; Mask, Hash, and Drop are not reversible. The map never touches disk here; encrypt it with encrypt_map and persistence stays with the caller.

Signature:

pub async fn redact_capturing_rehydration_map(result: &mut ExtractedDocument, config: &RedactionConfig) -> Result<RehydrationMap, XbergError>

Example:

let result = redact_capturing_rehydration_map(&mut ExtractedDocument::default(), &RedactionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&RedactionConfig`	Yes	The configuration options

Returns: RehydrationMap

Errors: Returns Err(XbergError).

redact_with_entities()

Redact result using config plus a caller-supplied entity stream, without invoking a NER backend.

redact is this function with the entities the backend configured in RedactionConfig::ner produced. Callers that already have an entity stream — from crate::enrich, from the NER post-processor, or from their own model — pass it here and skip the second inference pass. Entities of a category the pattern engine already covers (email, phone, URL) are ignored: the regex engine is the more reliable detector for those.

Signature:

pub fn redact_with_entities(result: &mut ExtractedDocument, config: &RedactionConfig, entities: &[Entity]) -> Result<ExtractedDocument, XbergError>

Example:

let result = redact_with_entities(&mut ExtractedDocument::default(), &RedactionConfig::default(), &vec![])?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&RedactionConfig`	Yes	The configuration options
`entities`	`&\[Entity\]`	Yes	The entities

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

parse_external_findings()

Parse findings from a JSON array or JSON Lines using the default safety cap.

Signature:

pub fn parse_external_findings(text: &str) -> Result<Vec<ExternalRedactionFinding>, XbergError>

Example:

let result = parse_external_findings("value")?;

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text

Returns: Vec<ExternalRedactionFinding>

Errors: Returns Err(XbergError).

parse_external_findings_bounded()

Parse at most max_findings findings from a JSON array or JSON Lines.

Signature:

pub fn parse_external_findings_bounded(text: &str, max_findings: u32) -> Result<Vec<ExternalRedactionFinding>, XbergError>

Example:

let result = parse_external_findings_bounded("value", 42)?;

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text
`max_findings`	`u32`	Yes	The max findings

Returns: Vec<ExternalRedactionFinding>

Errors: Returns Err(XbergError).

encrypt_map()

Encrypt map with passphrase. Returns XPII\x01 + salt(16) + nonce(12) + tag(16) + ciphertext.

Signature:

pub fn encrypt_map(map: &RehydrationMap, passphrase: &str) -> Result<Vec<u8>, XbergError>

Example:

let result = encrypt_map(&todo!(), "value")?;

Parameters:

Name	Type	Required	Description
`map`	`&RehydrationMap`	Yes	The rehydration map
`passphrase`	`&str`	Yes	The passphrase

Returns: Vec<u8>

Errors: Returns Err(XbergError).

decrypt_map()

Decrypt a blob from encrypt_map.

Signature:

pub fn decrypt_map(blob: &[u8], passphrase: &str) -> Result<RehydrationMap, XbergError>

Example:

let result = decrypt_map(b"data", "value")?;

Parameters:

Name	Type	Required	Description
`blob`	`&\[u8\]`	Yes	The blob
`passphrase`	`&str`	Yes	The passphrase

Returns: RehydrationMap

Errors: Returns Err(XbergError).

find_subject()

Search a decrypted map for query, matching either the token or the original value (case-insensitive substring match on original; exact match on token, since tokens are structured like "[EMAIL_1]").

Results are sorted by token for deterministic output.

Signature:

pub fn find_subject(map: &RehydrationMap, query: &str) -> Vec<SubjectMatch>

Example:

let result = find_subject(&todo!(), "value");

Parameters:

Name	Type	Required	Description
`map`	`&RehydrationMap`	Yes	The rehydration map
`query`	`&str`	Yes	The query

Returns: Vec<SubjectMatch>

forget_subject()

Remove every mapping whose token or original value matches query. Returns the removed entries (the caller re-encrypts and persists the resulting map; this function does not touch disk).

Idempotent: calling this again with the same query after the matching entries have already been removed returns an empty Vec.

Signature:

pub fn forget_subject(map: &mut RehydrationMap, query: &str) -> Vec<SubjectMatch>

Example:

let result = forget_subject(&mut todo!(), "value");

Parameters:

Name	Type	Required	Description
`map`	`&mut RehydrationMap`	Yes	The rehydration map
`query`	`&str`	Yes	The query

Returns: Vec<SubjectMatch>

find_unmarked_claims()

Find unmarked claims in markdown text.

Returns lines that assert a claim but carry neither a footnote citation anchor ([^...]) nor an inference marker ([*inference*]).

The heuristic is simple: a line that contains alphabetic words, ends with sentence punctuation, and is not a heading, blank line, or markup-only line is considered a claim. Exclude lines that appear in the citation block (after --- + ).

Returns:

A vector of trimmed line text strings for unmarked claims.

Signature:

pub fn find_unmarked_claims(markdown: &str) -> Vec<String>

Example:

let text = r#"This is a claim without citation.
Another claim with citation.[^1]
This is a claim with inference.[*inference*]

[^1]: Citation"#;
let unmarked = find_unmarked_claims(text);
assert_eq!(unmarked.len(), 1);
assert!(unmarked[0].contains("without citation"));

Parameters:

Name	Type	Required	Description
`markdown`	`&str`	Yes	The markdown text to search

Returns: Vec<String>

verify_excerpt()

Verify that an excerpt appears verbatim in source text.

Performs exact matching by default. Also tries whitespace-normalized matching (collapsing runs of whitespace on both sides) since PDF-extracted text often has irregular spacing.

Returns:

true if the excerpt appears (exactly or with normalized whitespace), false otherwise.

Signature:

pub fn verify_excerpt(excerpt: &str, source_text: &str) -> bool

Example:

let source = "The document states: Exact quoted text.";
let excerpt = "Exact quoted text";
assert!(verify_excerpt(excerpt, source));

// Whitespace normalization
let source2 = "Text with  irregular   spacing.";
let excerpt2 = "Text with irregular spacing";
assert!(verify_excerpt(excerpt2, source2));

Parameters:

Name	Type	Required	Description
`excerpt`	`&str`	Yes	The text snippet to find
`source_text`	`&str`	Yes	The full source text to search

Returns: bool

clean_extracted_text()

Apply the quality heuristics and return a cleaned representation of the text.

This function normalises whitespace, removes navigation boilerplate, and strips repeated punctuation that commonly appears in OCR output.

Signature:

pub fn clean_extracted_text(text: &str) -> String

Example:

let result = clean_extracted_text("value");

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text

Returns: String

openapi_json()

Generate OpenAPI JSON schema.

Returns the complete OpenAPI 3.1 specification as a JSON string.

Signature:

pub fn openapi_json() -> String

Example:

use xberg::api::openapi::openapi_json;

let schema = openapi_json();
println!("{}", schema);

Returns: String

create_router()

Create the API router with all routes configured.

This is public to allow users to embed the router in their own applications.

Signature:

pub fn create_router(config: ExtractionConfig) -> String

Example:

use xberg::{ExtractionConfig, api::create_router};

# #[tokio::main]
# async fn main() {
// Create router with default config and size limits
let config = ExtractionConfig::default();
let router = create_router(config);
# }

Parameters:

Name	Type	Required	Description
`config`	`ExtractionConfig`	Yes	Default extraction configuration. Per-request configs override these defaults.

Returns: String

create_router_with_limits()

Create the API router with custom size limits.

This allows fine-grained control over request body and multipart field size limits.

Signature:

pub fn create_router_with_limits(config: ExtractionConfig, limits: ApiSizeLimits) -> String

Example:

use xberg::{ExtractionConfig, api::{create_router_with_limits, ApiSizeLimits}};

# #[tokio::main]
# async fn main() {
// Create router with 50 MB limits
let config = ExtractionConfig::default();
let limits = ApiSizeLimits::from_mb(50, 50);
let router = create_router_with_limits(config, limits);
# }

use xberg::{ExtractionConfig, api::{create_router_with_limits, ApiSizeLimits}};
use tower_http::limit::RequestBodyLimitLayer;

# #[tokio::main]
# async fn main() {
// Custom limits for very large documents (500 MB)
let config = ExtractionConfig::default();
let limits = ApiSizeLimits::from_mb(500, 500);
let router = create_router_with_limits(config, limits);
# }

Parameters:

Name	Type	Required	Description
`config`	`ExtractionConfig`	Yes	Default extraction configuration. Per-request configs override these defaults.
`limits`	`ApiSizeLimits`	Yes	Size limits for request bodies and multipart uploads.

Returns: String

serve()

Start the API server with config file discovery.

Searches for xberg.toml/yaml/yml/json in current and parent directories. If no config file is found, uses default configuration.

Signature:

pub async fn serve(host: String, port: u16) -> Result<(), XbergError>

Example:

use xberg::api::serve;

#[tokio::main]
async fn main() -> xberg::Result<()> {
    // Local development
    serve("127.0.0.1", 8000).await?;
    Ok(())
}

use xberg::api::serve;

#[tokio::main]
async fn main() -> xberg::Result<()> {
    // Docker/production (listen on all interfaces)
    serve("0.0.0.0", 8000).await?;
    Ok(())
}

# Python/Docker usage
export XBERG_HOST=0.0.0.0
export XBERG_PORT=8000

# CORS configuration (IMPORTANT for production security)
# Default: allows all origins (permits CSRF attacks)
# Production: set to comma-separated list of allowed origins
export XBERG_CORS_ORIGINS="<https://app.example.com,https://api.example.com>"

# Upload size limits (default: 100 MB)
# Modern approach (in bytes):
export XBERG_MAX_REQUEST_BODY_BYTES=104857600       # 100 MB
export XBERG_MAX_MULTIPART_FIELD_BYTES=104857600    # 100 MB per file

python -m xberg.api

Parameters:

Name	Type	Required	Description
`host`	`String`	Yes	IP address to bind to (e.g., “127.0.0.1” or “0.0.0.0”)
`port`	`u16`	Yes	Port number to bind to (e.g., 8000)

Returns: No return value.

Errors: Returns Err(XbergError).

serve_with_config()

Start the API server with explicit config.

Uses default size limits (100 MB). For custom limits, use serve_with_config_and_limits.

Signature:

pub async fn serve_with_config(host: String, port: u16, config: ExtractionConfig) -> Result<(), XbergError>

Example:

use xberg::{ExtractionConfig, api::serve_with_config};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::from_toml_file("config/xberg.toml")?;
    serve_with_config("127.0.0.1", 8000, config).await?;
    Ok(())
}

Parameters:

Name	Type	Required	Description
`host`	`String`	Yes	IP address to bind to (e.g., “127.0.0.1” or “0.0.0.0”)
`port`	`u16`	Yes	Port number to bind to (e.g., 8000)
`config`	`ExtractionConfig`	Yes	Default extraction configuration for all requests

Returns: No return value.

Errors: Returns Err(XbergError).

serve_with_config_and_limits()

Start the API server with explicit config and size limits.

Signature:

pub async fn serve_with_config_and_limits(host: String, port: u16, config: ExtractionConfig, limits: ApiSizeLimits) -> Result<(), XbergError>

Example:

use xberg::{ExtractionConfig, api::{serve_with_config_and_limits, ApiSizeLimits}};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::from_toml_file("config/xberg.toml")?;
    let limits = ApiSizeLimits::from_mb(200, 200);
    serve_with_config_and_limits("127.0.0.1", 8000, config, limits).await?;
    Ok(())
}

Parameters:

Name	Type	Required	Description
`host`	`String`	Yes	IP address to bind to (e.g., “127.0.0.1” or “0.0.0.0”)
`port`	`u16`	Yes	Port number to bind to (e.g., 8000)
`config`	`ExtractionConfig`	Yes	Default extraction configuration for all requests
`limits`	`ApiSizeLimits`	Yes	Size limits for request bodies and multipart uploads

Returns: No return value.

Errors: Returns Err(XbergError).

serve_with_server_config()

Start the API server with explicit extraction config and server config.

This function accepts a fully-configured ServerConfig, including CORS origins, size limits, host, and port. It respects all ServerConfig fields without re-parsing environment variables, making it ideal for CLI usage where configuration precedence has already been applied.

Signature:

pub async fn serve_with_server_config(extraction_config: ExtractionConfig, server_config: ServerConfig) -> Result<(), XbergError>

Example:

use xberg::{ExtractionConfig, api::serve_with_server_config, core::ServerConfig};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let extraction_config = ExtractionConfig::default();
    let mut server_config = ServerConfig::default();
    server_config.host = "0.0.0.0".to_string();
    server_config.port = 3000;
    server_config.cors_origins = vec!["<https://example.com>".to_string()];

    serve_with_server_config(extraction_config, server_config).await?;
    Ok(())
}

Parameters:

Name	Type	Required	Description
`extraction_config`	`ExtractionConfig`	Yes	Default extraction configuration for all requests
`server_config`	`ServerConfig`	Yes	Server configuration including host, port, CORS, and size limits

Returns: No return value.

Errors: Returns Err(XbergError).

serve_default()

Start the API server with default host and port.

Defaults: host = “127.0.0.1”, port = 8000

Uses config file discovery (searches current/parent directories for xberg.toml/yaml/yml/json). Validates plugins at startup to help diagnose configuration issues.

Signature:

pub async fn serve_default() -> Result<(), XbergError>

Example:

serve_default().await?;

Returns: No return value.

Errors: Returns Err(XbergError).

resolve_extra_allowed_hosts()

Resolve the extra allowed hosts for the MCP HTTP transport using the precedence cascade: cli_hosts > env_value > config_hosts > default (empty).

The first non-empty tier wins in full; tiers are never merged with each other. Hosts are trimmed, empty entries are dropped, and duplicates are removed while preserving order. An empty result means rmcp’s built-in loopback-only default is used unchanged.

Signature:

pub fn resolve_extra_allowed_hosts(cli_hosts: &[String], env_value: Option<&str>, config_hosts: &[String]) -> Vec<String>

Example:

let result = resolve_extra_allowed_hosts(&vec![], Some("value"), &vec![]);

Parameters:

Name	Type	Required	Description
`cli_hosts`	`&\[String\]`	Yes	The cli hosts
`env_value`	`Option<&str>`	No	The env value
`config_hosts`	`&\[String\]`	Yes	The config hosts

Returns: Vec<String>

read_mcp_allowed_hosts_from_file()

Read the allowed_hosts list from an [mcp] table in a TOML, YAML, or JSON config file.

Returns an empty vector if the file has no [mcp] table or allowed_hosts key.

Errors:

Returns XbergError::Validation if the file cannot be read, has an unsupported extension, or contains invalid syntax for its detected format.

Signature:

pub fn read_mcp_allowed_hosts_from_file(path: PathBuf) -> Result<Vec<String>, XbergError>

Example:

let result = read_mcp_allowed_hosts_from_file("value")?;

Parameters:

Name	Type	Required	Description
`path`	`PathBuf`	Yes	Path to the file

Returns: Vec<String>

Errors: Returns Err(XbergError).

start_mcp_server()

Start the Xberg MCP server.

This function initializes and runs the MCP server using stdio transport. It will block until the server is shut down.

Errors:

Returns an error if the server fails to start or encounters a fatal error.

Signature:

pub async fn start_mcp_server() -> Result<(), Error + Send + Sync>>

Example:

use xberg::mcp::start_mcp_server;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    start_mcp_server().await?;
    Ok(())
}

Returns: No return value.

Errors: Returns Err(ErrorSendSync).

start_mcp_server_with_config()

Start MCP server with custom extraction config.

This variant allows specifying a custom extraction configuration (e.g., loaded from a file) instead of using defaults.

Signature:

pub async fn start_mcp_server_with_config(config: ExtractionConfig) -> Result<(), Error + Send + Sync>>

Example:

start_mcp_server_with_config(ExtractionConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`config`	`ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(ErrorSendSync).

start_mcp_server_http()

Start MCP server with HTTP Stream transport.

Uses rmcp’s built-in StreamableHttpService for HTTP/SSE support per MCP spec.

Signature:

pub async fn start_mcp_server_http(host: String, port: u16, extra_allowed_hosts: &[String]) -> Result<(), Error + Send + Sync>>

Example:

use xberg::mcp::start_mcp_server_http;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    start_mcp_server_http("127.0.0.1", 8001, &[]).await?;
    Ok(())
}

Parameters:

Name	Type	Required	Description
`host`	`String`	Yes	Host to bind to (e.g., “127.0.0.1” or “0.0.0.0”)
`port`	`u16`	Yes	Port number (e.g., 8001)
`extra_allowed_hosts`	`&\[String\]`	Yes	Additional `Host` header values to accept, on top of rmcp’s loopback-only default (`localhost`, `127.0.0.1`, `::1`). Needed when the server runs behind a reverse proxy or ingress that forwards a different hostname. Pass an empty slice to keep the default loopback-only behavior.

Returns: No return value.

Errors: Returns Err(ErrorSendSync).

start_mcp_server_http_with_config()

Start MCP HTTP server with custom extraction config.

This variant allows specifying a custom extraction configuration while using HTTP Stream transport.

Signature:

pub async fn start_mcp_server_http_with_config(host: String, port: u16, config: ExtractionConfig, extra_allowed_hosts: &[String]) -> Result<(), Error + Send + Sync>>

Example:

use xberg::mcp::start_mcp_server_http_with_config;
use xberg::ExtractionConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let config = ExtractionConfig::default();
    start_mcp_server_http_with_config("127.0.0.1", 8001, config, &[]).await?;
    Ok(())
}

Parameters:

Name	Type	Required	Description
`host`	`String`	Yes	Host to bind to (e.g., “127.0.0.1” or “0.0.0.0”)
`port`	`u16`	Yes	Port number (e.g., 8001)
`config`	`ExtractionConfig`	Yes	Custom extraction configuration
`extra_allowed_hosts`	`&\[String\]`	Yes	Additional `Host` header values to accept, on top of rmcp’s loopback-only default (`localhost`, `127.0.0.1`, `::1`). Needed when the server runs behind a reverse proxy or ingress that forwards a different hostname. Pass an empty slice to keep the default loopback-only behavior.

Returns: No return value.

Errors: Returns Err(ErrorSendSync).

chunk_text()

Split text into chunks with optional page boundary tracking.

This is the primary API function for chunking text. It supports both plain text and Markdown with configurable chunk size, overlap, and page boundary mapping.

Returns:

A ChunkingResult containing all chunks and their metadata.

Signature:

pub fn chunk_text(text: &str, config: &ChunkingConfig, page_boundaries: Option<&[PageBoundary]>) -> Result<String, XbergError>

Example:

use xberg::chunking::{chunk_text, ChunkingConfig, ChunkerType};

let config = ChunkingConfig {
    max_characters: 500,
    overlap: 50,
    trim: true,
    chunker_type: ChunkerType::Text,
    ..Default::default()
};
let result = chunk_text("Long text...", &config, None)?;
assert!(!result.chunks.is_empty());

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text to split into chunks
`config`	`&ChunkingConfig`	Yes	Chunking configuration (max size, overlap, type)
`page_boundaries`	`Option<&\[PageBoundary\]>`	No	Optional page boundary markers for mapping chunks to pages

Returns: String

Errors: Returns Err(XbergError).

render_heading_breadcrumb()

Render content with its Markdown heading breadcrumb prepended.

This is the explicit, consumer-applied rendering step described by the revised #1393 design: chunk_text never mutates chunk.content with a heading breadcrumb — content always stays exactly the source span (chunk.content == &source[byte_start..byte_end]). A caller that wants the breadcrumb inline — e.g. for the dense/embedding retrieval arm, which benefits from a self-contained passage that carries its own structural context — calls this function directly at index time with the chunk’s (clean) content and its heading_context, and keeps content itself untouched for BM25/SPLADE, which do not want the breadcrumb at all. See the rag module docs for the full retrieval-mode guidance.

Formats context’s heading hierarchy as a single breadcrumb line — each heading rendered as ATX hashes followed by its text, joined by " > " (e.g. "# Guide > ## Setup") — then a blank line, then content with a leading occurrence of the breadcrumb’s deepest (last) heading stripped so the heading is not duplicated when content itself already starts with that heading line, as Markdown-chunked content commonly does (the Markdown splitter keeps the heading that opens a section as the first line of that section’s first chunk).

Signature:

pub fn render_heading_breadcrumb(content: &str, context: &HeadingContext) -> String

Example:

use xberg::chunking::render_heading_breadcrumb;
use xberg::types::{HeadingContext, HeadingLevel};

let context = HeadingContext {
    headings: vec![
        HeadingLevel { level: 1, text: "Guide".to_string() },
        HeadingLevel { level: 2, text: "Setup".to_string() },
    ],
};
let rendered = render_heading_breadcrumb("## Setup\n\nInstall the dependencies.", &context);
assert_eq!(rendered, "# Guide > ## Setup\n\nInstall the dependencies.");

Parameters:

Name	Type	Required	Description
`content`	`&str`	Yes	The content to process
`context`	`&HeadingContext`	Yes	The heading context

Returns: String

count_tokens()

Count the number of tokens in text for the given HuggingFace tokenizer model.

Reuses the global in-memory tokenizer cache — the tokenizer is downloaded and parsed only on the first call for each model, then served from memory for all subsequent calls. File-level caching is handled by hf-hub, including its standard HF_HUB_CACHE, HUGGINGFACE_HUB_CACHE, HF_HOME, XDG, and platform cache conventions.

Returns:

The number of tokens produced by the tokenizer. If the tokenizer cannot be loaded (e.g. network unavailable, model not found), falls back to a whitespace-split heuristic so the function never panics or propagates an error.

Note:

This function is intentionally excluded from language bindings (alef-skipped) — it is a Rust-only utility that relies on the cached tokenizer infrastructure.

Signature:

pub fn count_tokens(text: &str, model: Option<&str>) -> usize

Example:

use xberg::chunking::count_tokens;

let n = count_tokens("Hello, world!", None);
assert!(n > 0);

let n_gpt4 = count_tokens("Hello, world!", Some("Xenova/gpt-4o"));
assert!(n_gpt4 > 0);

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text to tokenize.
`model`	`Option<&str>`	No	HuggingFace model ID to use for tokenization. Pass `None` to use the default model (`DEFAULT_COUNT_TOKENS_MODEL`, `"Xenova/gpt-4o"`), which encodes with the o200k_base vocabulary — the closest widely-available proxy for GPT-4o and Claude token counts.

Returns: usize

create_client_with_credential_provider()

Create a liter-llm ManagedClient from xberg’s LlmConfig, overriding any LlmConfig::credential_provider with an explicit, caller-supplied provider.

This is the escape hatch for authentication modes CredentialProviderConfig cannot express as data — GitHub Copilot’s interactive device-flow provider (liter_llm::auth::github_copilot::GithubCopilotCredentialProvider), or any fully custom liter_llm::auth::CredentialProvider implementation. Building provider requires the caller to depend on liter-llm directly; xberg does not re-export its auth types.

Deliberately not cached through LLM_CLIENT_POOL: the cache key is a digest of the serializable LlmConfig, and provider (an Arc<dyn CredentialProvider>) is not part of that — nor of any other serializable identity this function has access to. Two callers passing the same config with two different providers must never be handed the same client, so every call here builds a fresh, unpooled ManagedClient instead. It is still built through ManagedClient (via build_managed_client), so LlmConfig::max_concurrency still takes effect per client built this way — it simply is not shared across calls the way create_client’s cached client is.

Not available on wasm32: this function lives in crate::llm::client, which — like the rest of crate::llm — is compiled out on that target (see the crate-root pub mod llm; gate in lib.rs), because there is no native-http-backed CredentialProvider implementation to construct provider from there in the first place. LlmConfig::credential_provider itself still exists as a field on wasm32 (see that field’s docs) — it is simply never read.

This Rust-only function accepts a liter-llm trait object and is not exposed through language bindings.

Signature:

pub fn create_client_with_credential_provider(config: &LlmConfig, provider: CredentialProvider) -> Result<String, XbergError>

Example:

let result = create_client_with_credential_provider(&LlmConfig::default(), todo!())?;

Parameters:

Name	Type	Required	Description
`config`	`&LlmConfig`	Yes	The configuration options
`provider`	`CredentialProvider`	Yes	The provider

Returns: String

Errors: Returns Err(XbergError).

complete_with_json_schema()

Send a free-form prompt to the configured LLM with a JSON-schema response constraint and return the parsed JSON value plus captured usage.

This is the shared helper used by LLM-backed post-processors (page classification, LLM-driven NER, etc.) that need structured output but do not want to depend on StructuredExtractionConfig’s schema/prompt machinery.

Errors:

Returns an error if the LLM client cannot be constructed, the request fails, the response contains no content, or the response is not parseable JSON.

Signature:

pub async fn complete_with_json_schema(llm_config: &LlmConfig, prompt: &str, schema_name: &str, schema: &Value, source: &str) -> Result<String, XbergError>

Example:

let result = complete_with_json_schema(&LlmConfig::default(), "value", "value", &todo!(), "value").await?;

Parameters:

Name	Type	Required	Description
`llm_config`	`&LlmConfig`	Yes	The llm config
`prompt`	`&str`	Yes	The prompt
`schema_name`	`&str`	Yes	The schema name
`schema`	`&Value`	Yes	The value
`source`	`&str`	Yes	The source

Returns: String

Errors: Returns Err(XbergError).

complete_text()

Send a single user prompt to the configured LLM and return the response text along with the captured usage metadata.

The source argument labels the LlmUsage entry that is returned so callers can aggregate per-feature spend ("translation", "summarisation", etc.). The helper performs a single non-streaming chat completion request.

Errors:

Returns an error if the LLM client cannot be constructed, the request fails, or the response does not contain assistant content.

Signature:

pub async fn complete_text(llm_config: &LlmConfig, prompt: &str, source: &str) -> Result<String, XbergError>

Example:

let result = complete_text(&LlmConfig::default(), "value", "value").await?;

Parameters:

Name	Type	Required	Description
`llm_config`	`&LlmConfig`	Yes	The llm config
`prompt`	`&str`	Yes	The prompt
`source`	`&str`	Yes	The source

Returns: String

Errors: Returns Err(XbergError).

warm_model()

Eagerly download and cache an embedding model without returning the handle.

This triggers the same download and initialization as get_or_init_engine but discards the result, making it suitable for cache-warming scenarios where the caller doesn’t need to use the model immediately. Used internally by the api/mcp cache.warm endpoints and by the xberg-cli warm command. Excluded from the language bindings via alef.toml [exclude].functions.

Signature:

pub fn warm_model(model_type: &crate::core::config::EmbeddingModelType, cache_dir: Option<PathBuf>) -> Result<(), XbergError>

Example:

warm_model(&&crate::core::config::EmbeddingModelType::default(), Some("value"))?;

Parameters:

Name	Type	Required	Description
`model_type`	`&crate::core::config::EmbeddingModelType`	Yes	The embedding model type
`cache_dir`	`Option<PathBuf>`	No	The cache dir

Returns: No return value.

Errors: Returns Err(XbergError).

evict_model()

Drop every resident engine loaded for model so its memory can be freed.

Engines are matched by repository and model file, whatever pooling, sequence length, acceleration or cache directory they were loaded with. A caller that still holds an engine keeps it alive until it drops its handle. Returns the number of engines removed; Ok(0) means none was resident.

Errors:

Embedding for an unknown preset, or for an Llm or Plugin model, which keep no local engine.

Signature:

pub fn evict_model(model: &EmbeddingModelType) -> Result<usize, XbergError>

Example:

let result = evict_model(&EmbeddingModelType::default())?;

Parameters:

Name	Type	Required	Description
`model`	`&EmbeddingModelType`	Yes	The embedding model type

Returns: usize

Errors: Returns Err(XbergError).

clear_engine_cache()

Drop every resident embedding engine, ONNX and static alike. Returns the number of engines removed.

Signature:

pub fn clear_engine_cache() -> usize

Example:

let result = clear_engine_cache();

Returns: usize

set_engine_cache_limit()

Bound the number of embedding engines kept resident, or lift the bound with None.

The bound applies to each backend’s cache: at most max_resident ONNX engines and at most max_resident static (model2vec) engines stay loaded. When a new engine would exceed the bound, the least recently used one is dropped first. Lowering the bound drops engines at once. The default is no bound.

Signature:

pub fn set_engine_cache_limit(max_resident: Option<NonZeroUsize>)

Example:

set_engine_cache_limit(Some(todo!()));

Parameters:

Name	Type	Required	Description
`max_resident`	`Option<NonZeroUsize>`	No	The max resident

Returns: No return value.

evict_model()

Drop every resident reranker engine loaded for model so its memory can be freed.

Engines are matched by repository and model file, whatever sequence length, acceleration or cache directory they were loaded with. A caller that still holds an engine keeps it alive until it drops its handle. Returns the number of engines removed; Ok(0) means none was resident.

Errors:

Reranking for an unknown preset, or for an Llm or Plugin model, which keep no local engine.

Signature:

pub fn evict_model(model: &crate::core::config::RerankerModelType) -> Result<usize, XbergError>

Example:

let result = evict_model(&&crate::core::config::RerankerModelType::default())?;

Parameters:

Name	Type	Required	Description
`model`	`&crate::core::config::RerankerModelType`	Yes	The reranker model type

Returns: usize

Errors: Returns Err(XbergError).

set_ort_session_options()

Set the OrtSessionOptions used for ONNX sessions built from now on.

When the options actually change, the embedding, reranking, sparse-embedding and late-interaction engine caches are cleared so resident engines are rebuilt with the new options on next use.

Signature:

pub fn set_ort_session_options(options: OrtSessionOptions)

Example:

set_ort_session_options(OrtSessionOptions::default());

Parameters:

Name	Type	Required	Description
`options`	`OrtSessionOptions`	Yes	The options to use

Returns: No return value.

ort_session_options()

The OrtSessionOptions currently in effect.

Signature:

pub fn ort_session_options() -> OrtSessionOptions

Example:

let result = ort_session_options();

Returns: OrtSessionOptions

get_preset()

Look up a bundled SPLADE preset by exact name.

Signature:

pub fn get_preset(name: &str) -> Option<SparseEmbeddingPreset>

Example:

let result = get_preset("value");

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: Option<SparseEmbeddingPreset>

list_presets()

List the names of all bundled SPLADE presets.

Signature:

pub fn list_presets() -> Vec<String>

Example:

let result = list_presets();

Returns: Vec<String>

evict_model()

Drop every resident sparse-embedding engine loaded for model so its memory can be freed.

Errors:

Embedding for an unknown preset or a Plugin model.

Signature:

pub fn evict_model(model: &crate::core::config::SparseEmbeddingModelType) -> Result<usize, XbergError>

Example:

let result = evict_model(&&crate::core::config::SparseEmbeddingModelType::default())?;

Parameters:

Name	Type	Required	Description
`model`	`&crate::core::config::SparseEmbeddingModelType`	Yes	The sparse embedding model type

Returns: usize

Errors: Returns Err(XbergError).

embed_sparse_async()

Async wrapper over embed_sparse: runs the blocking ONNX inference on a bounded blocking-task pool so it does not stall the async runtime.

Signature:

pub async fn embed_sparse_async(texts: Vec<String>, config: &crate::core::config::SparseEmbeddingConfig) -> Result<Vec<SparseEmbedding>, XbergError>

Example:

let result = embed_sparse_async(vec![], &&crate::core::config::SparseEmbeddingConfig::default()).await?;

Parameters:

Name	Type	Required	Description
`texts`	`Vec<String>`	Yes	The texts
`config`	`&crate::core::config::SparseEmbeddingConfig`	Yes	The configuration options

Returns: Vec<SparseEmbedding>

Errors: Returns Err(XbergError).

get_preset()

Look up a bundled ColBERT preset by exact name.

Signature:

pub fn get_preset(name: &str) -> Option<LateInteractionPreset>

Example:

let result = get_preset("value");

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: Option<LateInteractionPreset>

max_sim_score()

Score a query against a document using ColBERT’s MaxSim operator: for each query token vector, take the maximum dot product against any document token vector, then sum across query tokens.

Returns 0.0 if query and doc have mismatched dimensionality, if either has zero tokens, or if either is not well-formed per MultiVectorEmbedding::is_well_formed (its data length does not match num_tokens * dim).

Pure CPU primitive — available without ONNX Runtime.

Signature:

pub fn max_sim_score(query: &MultiVectorEmbedding, doc: &MultiVectorEmbedding) -> f64

Example:

let result = max_sim_score(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`query`	`&MultiVectorEmbedding`	Yes	The multi vector embedding
`doc`	`&MultiVectorEmbedding`	Yes	The multi vector embedding

Returns: f64

max_sim_rank()

Rank a set of documents against a query by MaxSim score, descending.

Mirrors the sort/truncate shape of crate::reranking’s build_results, minus top-k truncation (callers slice the returned Vec themselves).

Pure CPU primitive — available without ONNX Runtime.

Signature:

pub fn max_sim_rank(query: &MultiVectorEmbedding, docs: &[MultiVectorEmbedding]) -> Vec<LateInteractionMatch>

Example:

let result = max_sim_rank(&todo!(), &vec![]);

Parameters:

Name	Type	Required	Description
`query`	`&MultiVectorEmbedding`	Yes	The multi vector embedding
`docs`	`&\[MultiVectorEmbedding\]`	Yes	The docs

Returns: Vec<LateInteractionMatch>

evict_model()

Drop every resident late-interaction engine loaded for model so its memory can be freed.

Engines are matched by repository and model file, whatever sequence lengths, acceleration or cache directory they were loaded with. A caller that still holds an engine keeps it alive until it drops its handle. Returns the number of engines removed; Ok(0) means none was resident.

Errors:

Embedding for an unknown preset or a Plugin model.

Signature:

pub fn evict_model(model: &crate::core::config::LateInteractionModelType) -> Result<usize, XbergError>

Example:

let result = evict_model(&&crate::core::config::LateInteractionModelType::default())?;

Parameters:

Name	Type	Required	Description
`model`	`&crate::core::config::LateInteractionModelType`	Yes	The late interaction model type

Returns: usize

Errors: Returns Err(XbergError).

embed_multi_vector_async()

Async wrapper over embed_multi_vector: runs the blocking ONNX inference on a bounded blocking-task pool so it does not stall the async runtime.

Signature:

pub async fn embed_multi_vector_async(texts: Vec<String>, config: &crate::core::config::LateInteractionConfig, is_query: bool) -> Result<Vec<MultiVectorEmbedding>, XbergError>

Example:

let result = embed_multi_vector_async(vec![], &&crate::core::config::LateInteractionConfig::default(), true).await?;

Parameters:

Name	Type	Required	Description
`texts`	`Vec<String>`	Yes	The texts
`config`	`&crate::core::config::LateInteractionConfig`	Yes	The configuration options
`is_query`	`bool`	Yes	The is query

Returns: Vec<MultiVectorEmbedding>

Errors: Returns Err(XbergError).

compute_hash()

Compute a blake3 hash string from input data.

Returns a 32-character hex string (128 bits of blake3 output).

Signature:

pub fn compute_hash(data: &str) -> String

Example:

let result = compute_hash("value");

Parameters:

Name	Type	Required	Description
`data`	`&str`	Yes	The data

Returns: String

validate_language_code()

Validate a Tesseract language code (or +-separated list of codes).

Accepts "all" and "*" as special values that auto-detect installed languages.

Errors:

Returns OcrError::InvalidLanguageCode if any component is not in the supported set.

Signature:

pub fn validate_language_code(lang_code: &str) -> Result<(), OcrError>

Example:

validate_language_code("value")?;

Parameters:

Name	Type	Required	Description
`lang_code`	`&str`	Yes	The lang code

Returns: No return value.

Errors: Returns Err(OcrError).

clean_obsolete()

Remove obsolete regular files directly inside the xberg cache directory. Directories and symlinks are never removed. Returns None without touching anything when XBERG_CACHE_DIR is set: ownership of an overridden base cannot be verified, so it is never cleaned.

Excluded from the polyglot binding surface: exposed for the CLI’s xberg doctor --clean.

Signature:

pub fn clean_obsolete() -> Option<CleanOutcome>

Example:

let result = clean_obsolete();

Returns: Option<CleanOutcome>

doctor()

Probe the backends and settings in config and report what will actually execute on this host.

Runs no downloads and no billable API calls. Backends that are not compiled in or whose models are not cached report Skip rather than failing.

Signature:

pub fn doctor(config: &ExtractionConfig) -> DoctorReport

Example:

let result = doctor(&ExtractionConfig::default());

Parameters:

Name	Type	Required	Description
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: DoctorReport

seed_classic_ocr_cache()

Download and verify every PaddleOCR and Sceptre model supported by this build.

The explicit cache_dir is used only for these classical OCR model families. Entries are status-free and sorted, so repeated successful calls against the same directory serialize identically.

Signature:

pub fn seed_classic_ocr_cache(cache_dir: PathBuf) -> Result<ClassicOcrCacheManifest, XbergError>

Example:

let result = seed_classic_ocr_cache("value")?;

Parameters:

Name	Type	Required	Description
`cache_dir`	`PathBuf`	Yes	The cache dir

Returns: ClassicOcrCacheManifest

Errors: Returns Err(XbergError).

manifest()

The manifest for cache manifest / MCP model listings.

Signature:

pub fn manifest() -> Vec<ModelManifestEntry>

Example:

let result = manifest();

Returns: Vec<ModelManifestEntry>

models_cached_in()

True when every model file is already cached in dir (default cache when None).

Signature:

pub fn models_cached_in(dir: Option<&PathBuf>) -> bool

Example:

let result = models_cached_in(Some("value"));

Parameters:

Name	Type	Required	Description
`dir`	`Option<&PathBuf>`	No	The dir

Returns: bool

models_cached()

True when every model file is already cached in the default location.

Signature:

pub fn models_cached() -> bool

Example:

let result = models_cached();

Returns: bool

ensure_models_in()

Download (if needed) and verify the model set in dir (default cache when None), returning the local paths. Per-file downloads serialize through the shared in-process download lock.

Signature:

pub fn ensure_models_in(dir: Option<&PathBuf>) -> Result<FormulaModelPaths, String>

Example:

let result = ensure_models_in(Some("value"))?;

Parameters:

Name	Type	Required	Description
`dir`	`Option<&PathBuf>`	No	The dir

Returns: FormulaModelPaths

Errors: Returns Err(String).

ensure_models()

Download (if needed) and verify the model set in the default location.

Signature:

pub fn ensure_models() -> Result<FormulaModelPaths, String>

Example:

let result = ensure_models()?;

Returns: FormulaModelPaths

Errors: Returns Err(String).

recognize_for_test()

Test-only public entry: the integration test exercises the full download-load-recognize pipeline through this.

Signature:

pub fn recognize_for_test(crop: &RgbImage) -> Result<Option<String>, String>

Example:

let result = recognize_for_test(&todo!())?;

Parameters:

Name	Type	Required	Description
`crop`	`&RgbImage`	Yes	The crop

Returns: Option<String>

Errors: Returns Err(String).

pdf_page_count()

Count the pages in a PDF without rendering any of them.

Opens the document and returns its page count from the PDF structure. No page is rasterized, so this is cheap relative to render_pdf_page_to_png — use it when you only need the count (e.g. to drive a render loop over the pages).

Errors:

Returns XbergError::Parsing if the PDF cannot be opened, authenticated, or its page count read.

Signature:

pub fn pdf_page_count(pdf_bytes: &[u8], password: Option<&str>) -> Result<usize, XbergError>

Example:

let result = pdf_page_count(b"data", Some("value"))?;

Parameters:

Name	Type	Required	Description
`pdf_bytes`	`&\[u8\]`	Yes	Raw PDF file bytes
`password`	`Option<&str>`	No	Optional password for encrypted PDFs

Returns: usize

Errors: Returns Err(XbergError).

decode_audio_to_pcm()

Decode arbitrary audio bytes (mp3, wav, m4a, webm, etc.) into 16 kHz mono f32 PCM.

This is a blocking CPU-heavy operation — callers should use tokio::task::spawn_blocking when on an async runtime.

Signature:

pub fn decode_audio_to_pcm(bytes: &[u8], max_bytes: Option<u64>) -> Result<PcmAudio, XbergError>

Example:

let result = decode_audio_to_pcm(b"data", Some(42))?;

Parameters:

Name	Type	Required	Description
`bytes`	`&\[u8\]`	Yes	The bytes
`max_bytes`	`Option<u64>`	No	The max bytes

Returns: PcmAudio

Errors: Returns Err(XbergError).

ensure_whisper_model()

Resolve a Whisper model to on-disk ONNX paths, downloading from HF Hub if needed.

Behaviour

Resolve every required file at the model’s immutable revision through hf-hub’s snapshot cache.
If allow_network is false, inspect only existing cache entries.
If any required file is absent in cache-only mode, return WhisperModelError::ModelMissing.
Return the hf-hub snapshot paths directly without copying model bytes.

Errors:

Returns WhisperModelError on I/O failures, download failures, or when the model is unavailable and allow_network is false.

Signature:

pub fn ensure_whisper_model(model: WhisperModel, cache_dir: Option<&PathBuf>, allow_network: bool, verify_hash: bool) -> Result<WhisperModelPaths, WhisperModelError>

Example:

let result = ensure_whisper_model(WhisperModel::Tiny, Some("value"), true, true)?;

Parameters:

Name	Type	Required	Description
`model`	`WhisperModel`	Yes	The whisper model
`cache_dir`	`Option<&PathBuf>`	No	The cache dir
`allow_network`	`bool`	Yes	The allow network
`verify_hash`	`bool`	Yes	The verify hash

Returns: WhisperModelPaths

Errors: Returns Err(WhisperModelError).

read_audio_tags()

Attempt to read audio tags and properties from raw bytes.

Never panics or returns an error — failures produce AudioTags::default().

Signature:

pub fn read_audio_tags(bytes: &[u8]) -> AudioTags

Example:

let result = read_audio_tags(b"data");

Parameters:

Name	Type	Required	Description
`bytes`	`&\[u8\]`	Yes	The bytes

Returns: AudioTags

clear_engine_caches()

Drop every resident model engine: dense, static, sparse and late-interaction embeddings and rerankers. Returns the number of engines removed.

A caller that still holds an engine keeps it alive until it drops its handle. The next call that needs a model loads it again. A build without any engine feature has nothing to drop and returns 0.

Signature:

pub fn clear_engine_caches() -> usize

Example:

let result = clear_engine_caches();

Returns: usize

embedding_query_prefix()

Query-side instruction prefix for an embedding config, if its preset defines one (asymmetric retrieval models such as Arctic-Embed). The RAG query path prepends this to query text; document text is embedded verbatim. Returns None for symmetric presets, custom models, and non-preset backends.

Signature:

pub fn embedding_query_prefix(config: &EmbeddingConfig) -> Option<String>

Example:

let result = embedding_query_prefix(&EmbeddingConfig::default());

Parameters:

Name	Type	Required	Description
`config`	`&EmbeddingConfig`	Yes	The configuration options

Returns: Option<String>

embed_sparse()

Generate sparse (SPLADE) embeddings for a list of texts.

Returns one SparseEmbedding per input text, in order.

Signature:

pub fn embed_sparse(texts: Vec<String>, config: &core::config::SparseEmbeddingConfig) -> Result<Vec<SparseEmbedding>, XbergError>

Example:

let result = embed_sparse(vec![], &&core::config::SparseEmbeddingConfig::default())?;

Parameters:

Name	Type	Required	Description
`texts`	`Vec<String>`	Yes	The texts
`config`	`&core::config::SparseEmbeddingConfig`	Yes	The configuration options

Returns: Vec<SparseEmbedding>

Errors: Returns Err(XbergError).

get_sparse_embedding_preset()

Get a sparse-embedding preset by name.

Signature:

pub fn get_sparse_embedding_preset(name: &str) -> Option<SparseEmbeddingPreset>

Example:

let result = get_sparse_embedding_preset("value");

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: Option<SparseEmbeddingPreset>

list_sparse_embedding_presets()

List the names of all available sparse-embedding presets.

Signature:

pub fn list_sparse_embedding_presets() -> Vec<String>

Example:

let result = list_sparse_embedding_presets();

Returns: Vec<String>

embed_multi_vector()

Generate ColBERT multi-vector embeddings for a list of texts.

is_query selects [Q]/[D] marker insertion and, when true, query augmentation padding.

Signature:

pub fn embed_multi_vector(texts: Vec<String>, config: &core::config::LateInteractionConfig, is_query: bool) -> Result<Vec<MultiVectorEmbedding>, XbergError>

Example:

let result = embed_multi_vector(vec![], &&core::config::LateInteractionConfig::default(), true)?;

Parameters:

Name	Type	Required	Description
`texts`	`Vec<String>`	Yes	The texts
`config`	`&core::config::LateInteractionConfig`	Yes	The configuration options
`is_query`	`bool`	Yes	The is query

Returns: Vec<MultiVectorEmbedding>

Errors: Returns Err(XbergError).

get_late_interaction_preset()

Get a late-interaction preset by name.

Signature:

pub fn get_late_interaction_preset(name: &str) -> Option<LateInteractionPreset>

Example:

let result = get_late_interaction_preset("value");

Parameters:

Name	Type	Required	Description
`name`	`&str`	Yes	The name

Returns: Option<LateInteractionPreset>

list_late_interaction_presets()

List the names of all available late-interaction presets.

Signature:

pub fn list_late_interaction_presets() -> Vec<String>

Example:

let result = list_late_interaction_presets();

Returns: Vec<String>

Types

AccelerationConfig

Hardware acceleration configuration for ONNX Runtime models.

Controls which execution provider (CPU, CoreML, CUDA, TensorRT) is used for inference in layout detection and embedding generation.

Field	Type	Default	Description
`provider`	`ExecutionProviderType`	`ExecutionProviderType::Auto`	Execution provider to use for ONNX inference.
`device_id`	`u32`	—	GPU device ID (for CUDA/TensorRT). Ignored for CPU/CoreML/Auto.

AnchorProperties

Properties for anchored drawings.

Field	Type	Default	Description
`behind_doc`	`bool`	—	Whether the drawing is placed behind the document text.
`layout_in_cell`	`bool`	—	Whether the drawing is laid out inside a table cell.
`relative_height`	`Option<i64>`	`Default::default()`	Z-order relative height used for stacking overlapping objects.
`position_h`	`Option<Position>`	`Default::default()`	Horizontal position specification.
`position_v`	`Option<Position>`	`Default::default()`	Vertical position specification.
`wrap_type`	`WrapType`	`WrapType::None`	Text-wrapping mode around this drawing.

ApiDoc

OpenAPI documentation structure.

Defines all endpoints, request/response schemas, and examples for the Xberg document extraction API.

ApiError

API-specific error wrapper.

Field	Type	Default	Description
`status`	`StatusCode`	—	HTTP status code
`body`	`ErrorResponse`	—	Error response body

Methods

into_response()

Signature:

pub fn into_response(self) -> String

Example:

let result = instance.into_response();

Returns: String

ApiSizeLimits

API server size limit configuration.

Controls maximum sizes for request bodies and multipart uploads. Default limits are set to 100 MB to accommodate typical document processing workloads.

Default Values

max_request_body_bytes: 100 MB (104,857,600 bytes)
max_multipart_field_bytes: 100 MB (104,857,600 bytes)

Configuration via Environment Variables

You can override the defaults using these environment variables:

# Modern approach (in bytes):
export XBERG_MAX_REQUEST_BODY_BYTES=104857600     # 100 MB
export XBERG_MAX_MULTIPART_FIELD_BYTES=104857600  # 100 MB

Field	Type	Default	Description
`max_request_body_bytes`	`usize`	`104857600`	Maximum size of the entire request body in bytes. This applies to the total size of all uploaded files and form data in a single request. Default: 100 MB (104,857,600 bytes).
`max_multipart_field_bytes`	`usize`	`104857600`	Maximum size of a single multipart field in bytes. This applies to individual files in a multipart upload. Default: 100 MB (104,857,600 bytes).

Methods

default()

Signature:

pub fn default() -> ApiSizeLimits

Example:

let result = ApiSizeLimits::default();

Returns: ApiSizeLimits

ApiState

API server state.

Holds the default extraction configuration loaded from config file (via discovery or explicit path). Per-request configs override these defaults.

Field	Type	Default	Description
`default_config`	`ExtractionConfig`	—	Default extraction configuration
`extraction_service`	`BoxCloneService`	—	Tower service for extraction requests. Wrapped in `Arc<Mutex>` because `BoxCloneService` is `Send` but not `Sync`, while `ApiState` must be `Clone + Sync` for Axum’s state requirement. The lock is held only long enough to clone the service.
`job_store`	`JobStore`	—	In-memory job store for async extraction polling.
`job_timeout_secs`	`u64`	—	Fallback timeout, in seconds, applied to a `POST /extract-async` job whose effective config does not pin down `extraction_timeout_secs` — copied from `ServerConfig::job_timeout_secs` when the router is built. A per-request `extraction_timeout_secs: Some(n)` always overrides this value; an explicit `extraction_timeout_secs: null` still falls back to it (see `ServerConfig::job_timeout_secs` for why an explicit `null` is deliberately not “unbounded”).

ArchiveArchiveEntry

Information about a single file in an archive.

Field	Type	Default	Description
`path`	`String`	—	File path within the archive, copied verbatim from the archive’s own entry name (`zip::read::ZipFile::name()`, `tar::Entry::path()`, …). This value is untrusted and unnormalised: a hostile or malformed archive can make it an absolute path (`/etc/passwd`), a traversing relative path (`../../etc/passwd`), or a Windows drive-letter/UNC form. Nothing in this module rejects or rewrites those forms, because nothing here writes archive contents to a real filesystem path built from this string – it is only ever used as an opaque map key (`extract_zip_text_content`/`extract_zip_file_bytes` and their TAR equivalents) or surfaced as informational metadata. A future caller that does want to write an entry to disk (a CLI “extract to directory” feature, an FFI binding, …) must never join this value onto a real path directly. Use `ArchiveEntry::confined_path` instead, which returns a normalised path guaranteed not to escape the archive root, or `None` when the raw name cannot be confined at all.
`size`	`u64`	—	File size in bytes
`is_dir`	`bool`	—	Whether this is a directory

ArchiveArchiveMetadata

Archive metadata extracted from an archive file.

Field	Type	Default	Description
`format`	`String`	—	Archive format (e.g., “ZIP”, “TAR”)
`file_list`	`Vec<ArchiveEntry>`	—	List of files in the archive
`file_count`	`usize`	—	Total number of files
`total_size`	`u64`	—	Total uncompressed size in bytes

ArchiveEntry

A single file extracted from an archive.

When archives (ZIP, TAR, 7Z, GZIP) are extracted with recursive extraction enabled, each processable file produces its own full ExtractedDocument.

Field	Type	Default	Description
`path`	`String`	—	Archive-relative file path (e.g. “folder/document.pdf”).
`mime_type`	`String`	—	Detected MIME type of the file.
`result`	`ExtractedDocument`	—	Full extraction result for this file.

ArchiveMetadata

Archive (ZIP/TAR/7Z) metadata.

Extracted from compressed archive files containing file lists and size information.

Field	Type	Default	Description
`format`	`String`	—	Archive format (“ZIP”, “TAR”, “7Z”, etc.)
`file_count`	`u32`	—	Total number of files in the archive
`file_list`	`Vec<String>`	`vec![]`	List of file paths within the archive
`total_size`	`u64`	—	Total uncompressed size in bytes
`compressed_size`	`Option<u64>`	`Default::default()`	Compressed size in bytes (if available)

AsciiDocExtractor

AsciiDoc document extractor.

Produces headings, lists, code blocks, tables, admonitions and quotes from AsciiDoc source, together with TextMetadata carrying the document’s headers, links and code blocks.

Methods

default()

Signature:

pub fn default() -> AsciiDocExtractor

Example:

let result = AsciiDocExtractor::default();

Returns: AsciiDocExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Attributes

Element attributes in Djot.

Represents the attributes attached to elements using {.class #id key=“value”} syntax.

Field	Type	Default	Description
`id`	`Option<String>`	`Default::default()`	Element ID (#identifier)
`classes`	`Vec<String>`	`vec![]`	CSS classes (.class1 .class2)
`key_values`	`Vec<KeyValueAttribute>`	`vec![]`	Key-value pairs (key=“value”)

AudioMetadata

Audio/video file metadata.

Populated from container tags (ID3v2, MP4 atoms, Vorbis comments, etc.) and PCM decode properties. Available when the transcription-types feature is enabled.

Field	Type	Default	Description
`duration_ms`	`Option<u64>`	`Default::default()`	Duration in milliseconds derived from the decoded audio stream.
`codec`	`Option<String>`	`Default::default()`	Audio codec (e.g. “mp3”, “aac”, “opus”, “flac”).
`container`	`Option<String>`	`Default::default()`	Container format (e.g. “mpeg”, “mp4”, “ogg”, “wav”).
`sample_rate_hz`	`Option<u32>`	`Default::default()`	Sample rate in Hz after decode (always 16000 when resampled for Whisper).
`channels`	`Option<u16>`	`Default::default()`	Number of audio channels (1 = mono, 2 = stereo).
`bitrate`	`Option<u32>`	`Default::default()`	Audio bitrate in kbps from the source file tags/properties.

AudioTags

Tag and audio-property data extracted from an audio/video file.

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Track title (`TIT2` / `\xa9nam` / Vorbis `TITLE`).
`artist`	`Option<String>`	`Default::default()`	Artist/performer — maps to `Metadata::authors` (wrapped in a Vec).
`year`	`Option<String>`	`Default::default()`	Release year as a four-digit string (e.g. “2023”) for `Metadata::created_at`.
`language`	`Option<String>`	`Default::default()`	ISO 639 language tag if present in the file (e.g. “eng”, “deu”).
`duration_ms`	`Option<u64>`	`Default::default()`	Duration in milliseconds from the file’s audio properties.
`sample_rate_hz`	`Option<u32>`	`Default::default()`	Sample rate in Hz from the file’s audio properties.
`channels`	`Option<u16>`	`Default::default()`	Channel count from the file’s audio properties.
`bitrate`	`Option<u32>`	`Default::default()`	Overall bitrate in kbps from the file’s audio properties.
`container`	`Option<String>`	`Default::default()`	Container format string derived from lofty’s `FileType`.

BBox

Bounding box in original image coordinates (x1, y1) top-left, (x2, y2) bottom-right.

Field	Type	Default	Description
`x1`	`f32`	—	Left edge (x-coordinate of the top-left corner).
`y1`	`f32`	—	Top edge (y-coordinate of the top-left corner).
`x2`	`f32`	—	Right edge (x-coordinate of the bottom-right corner).
`y2`	`f32`	—	Bottom edge (y-coordinate of the bottom-right corner).

BatchItemResult

Result for a single item in a batch OCR operation.

Field	Type	Default	Description
`file_path`	`String`	—	Path to the source file that was processed.
`success`	`bool`	—	Whether OCR succeeded for this file.
`result`	`Option<OcrExtractionResult>`	`None`	Extraction result, present when `success` is `true`.
`error`	`Option<String>`	`None`	Error message, present when `success` is `false`.

BedrockConfig

Since: v1.1

AWS Bedrock configuration for bedrock/-prefixed models.

Mirrors liter-llm’s BedrockConfig. Every field is optional: anything left unset falls back to the standard AWS environment variables (AWS_DEFAULT_REGION / AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, BEDROCK_CROSS_REGION) and the default AWS credential chain. Leave the credential fields unset unless you have an explicit reason to pin them.

Debug is implemented by hand so the three credential fields are never printed.

Field	Type	Default	Description
`region`	`Option<String>`	`Default::default()`	AWS region (e.g. `"us-east-1"`).
`cross_region_prefix`	`Option<String>`	`Default::default()`	Cross-region inference profile prefix (e.g. `"us"`).
`access_key_id`	`Option<String>`	`Default::default()`	Explicit AWS access key ID. Secret — never logged.
`secret_access_key`	`Option<String>`	`Default::default()`	Explicit AWS secret access key. Secret — never logged.
`session_token`	`Option<String>`	`Default::default()`	Explicit AWS session token for temporary credentials. Secret — never logged.

BibtexExtractor

BibTeX bibliography extractor.

Parses BibTeX files and extracts structured bibliography data including entries, authors, publication years, and entry type distribution.

Methods

default()

Signature:

pub fn default() -> BibtexExtractor

Example:

let result = BibtexExtractor::default();

Returns: BibtexExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

BibtexMetadata

BibTeX bibliography metadata.

Field	Type	Default	Description
`entry_count`	`usize`	—	Number of entries in the bibliography.
`citation_keys`	`Vec<String>`	`vec![]`	BibTeX citation keys (e.g. `"knuth1984"`) for all entries.
`authors`	`Vec<String>`	`vec![]`	Author names collected across all bibliography entries.
`year_range`	`Option<YearRange>`	`Default::default()`	Earliest and latest publication years found in the bibliography.
`entry_types`	`Option<HashMap<String, usize>>`	`HashMap::new()`	Count of entries grouped by BibTeX entry type (e.g. `"article"` → 5).

BorderStyle

A single border specification.

Field	Type	Default	Description
`style`	`String`	—	Border line style: `"single"`, `"double"`, `"dashed"`, `"dotted"`, `"none"`, etc.
`size`	`Option<i32>`	`None`	Border thickness in eighths of a point.
`color`	`Option<String>`	`None`	Border color as a hex RGB string (e.g. `"2F5496"`) or `"auto"`.
`space`	`Option<i32>`	`None`	Spacing between the border and the cell contents in points.

BoundingBox

Bounding box coordinates for element positioning.

Field	Type	Default	Description
`x0`	`f64`	—	Left x-coordinate
`y0`	`f64`	—	Bottom y-coordinate
`x1`	`f64`	—	Right x-coordinate
`y1`	`f64`	—	Top y-coordinate

BrowserConfig

Browser fallback configuration.

Field	Type	Default	Description
`mode`	`BrowserMode`	`BrowserMode::Auto`	When to use the headless browser fallback.
`backend`	`BrowserBackend`	`BrowserBackend::Chromiumoxide`	Browser backend used to render JavaScript-heavy pages.
`endpoint`	`Option<String>`	`None`	CDP WebSocket endpoint for connecting to an external browser instance. Crawlberg disconnects during teardown but never closes the external browser process.
`timeout`	`std::time::Duration`	`30000ms`	Timeout for browser page load and rendering (in milliseconds when serialized).
`overall_timeout`	`std::time::Duration`	`60000ms`	Overall deadline for a single browser fetch, covering browser launch (or page acquisition from a shared pool), page setup, navigation, rendering, and screenshot capture. Must exceed `timeout` to leave room for launch and setup overhead; a fetch that has not returned within this deadline fails with a timeout error (in milliseconds when serialized). Shutdown/teardown is governed separately by `shutdown_timeout` and is not counted against this deadline: an already-computed result is delivered to the caller without waiting for the browser process to exit.
`shutdown_timeout`	`std::time::Duration`	`5000ms`	How long to wait for the browser process to close and exit cleanly during teardown before the process is forcibly killed (in milliseconds when serialized).
`wait`	`BrowserWait`	`BrowserWait::NetworkIdle`	Wait strategy after browser navigation.
`wait_selector`	`Option<String>`	`None`	CSS selector to wait for when `wait` is `Selector`.
`extra_wait`	`Option<std::time::Duration>`	`None`	Extra time to wait after the wait condition is met.
`proxy`	`Option<ProxyConfig>`	`None`	Proxy for browser fetches. Overrides `CrawlConfig.proxy` when set. Native backend supports http/https only (no SOCKS5).
`block_url_patterns`	`Vec<String>`	`vec![]`	URL patterns to block before the network request fires. Supports `*` wildcards. Useful for skipping ads/analytics/large images. Honored by `BrowserBackend::Native`; chromiumoxide ignores this field today.
`eval_script`	`Option<String>`	`None`	JavaScript snippet evaluated after navigation completes. Scraping captures the native backend result in `ScrapeResult.browser.eval_result`. Interactions run this script before page actions on both browser backends but do not include the script result in `InteractionResult`.
`robots_user_agent`	`Option<String>`	`None`	User-agent used when fetching robots.txt. Defaults to `BrowserConfig.user_agent` (or crawlberg’s default) if unset. Native only.
`capture_network_events`	`bool`	`false`	Capture the full network event stream into the result. Default false (only the document event is captured). Native only.
`session_affinity`	`bool`	`true`	Enable session affinity: reuse chromiumoxide Pages for same-domain requests so cookies + fingerprint + solved challenges persist. Default: true. When false, each request gets a fresh Page.
`chrome_path`	`Option<PathBuf>`	`None`	Chrome or Chromium executable to launch. When set, crawlberg launches only this binary, and a path that is missing or not executable is an error that names the path; crawlberg never falls back to a different Chrome. When unset, crawlberg uses the `CHROME` environment variable, then searches the machine for an installed Chrome, Chromium or Edge. Chromiumoxide backend only: ignored, with a warning, when `endpoint` is set, with the native backend, and by scrapes and crawls that use a shared browser pool.
`chrome_args`	`Vec<String>`	`vec![]`	Extra Chrome command-line flags, each written as `--flag` or `--flag=value`, for example `--user-agent=...`. A flag here replaces a crawlberg default flag of the same name (`--lang=fr` replaces crawlberg’s `--lang=en_US`). Rejected: an entry that does not start with `--`, a flag name with an uppercase letter (Chrome flag names are lowercase), a flag named twice, and `--headless`, `--remote-debugging-port` and `--user-data-dir`, which crawlberg sets itself to run Chrome. Set this only from trusted configuration, like `proxy`: flags such as `--proxy-server` and `--host-resolver-rules` send Chrome’s traffic around the `ssrf` policy. Chromiumoxide backend only: ignored, with a warning, when `endpoint` is set, with the native backend, and by scrapes and crawls that use a shared browser pool.

CacheClearResponse

Xberg-managed cache clear response.

Shared Hugging Face Hub cache files are not included in this operation.

Field	Type	Default	Description
`directory`	`String`	—	Xberg-managed cache directory path
`removed_files`	`usize`	—	Number of files removed
`freed_mb`	`f64`	—	Space freed in MB

CacheStats

Aggregate statistics for a xberg cache directory.

Field	Type	Default	Description
`total_files`	`usize`	—	Total number of files currently in the cache directory.
`total_size_mb`	`f64`	—	Combined size of all cache files in megabytes.
`available_space_mb`	`f64`	—	Free disk space available on the cache volume, in megabytes.
`oldest_file_age_days`	`f64`	—	Age of the oldest cache file in days (0.0 if the cache is empty).
`newest_file_age_days`	`f64`	—	Age of the most recently written cache file in days (0.0 if the cache is empty).

CacheStatsResponse

Cache statistics response.

Field	Type	Default	Description
`directory`	`String`	—	Cache directory path
`total_files`	`usize`	—	Total number of cache files
`total_size_mb`	`f64`	—	Total cache size in MB
`available_space_mb`	`f64`	—	Available disk space in MB
`oldest_file_age_days`	`f64`	—	Age of oldest file in days
`newest_file_age_days`	`f64`	—	Age of newest file in days

CacheWarmParams

Request parameters for cache warm (model download).

Field	Type	Default	Description
`all_embeddings`	`bool`	`/* serde(default) */`	Download all embedding model presets
`embedding_model`	`Option<String>`	`None`	Specific embedding preset name to download (e.g. “balanced”, “speed”, “quality”)
`ner`	`bool`	`/* serde(default) */`	Download the default GLiNER NER model into the standard Hugging Face cache
`all_ner_models`	`bool`	`/* serde(default) */`	Download every known GLiNER NER model into the standard Hugging Face cache
`ner_model`	`Option<String>`	`None`	Specific GLiNER NER model alias or catalog id to download into the standard Hugging Face cache

CancellationToken

A lightweight, cloneable, one-shot cancellation token.

Create one with CancellationToken::new, pass one clone to extraction through ExtractionConfig::cancel_token, and retain another clone in the caller. Calling cancel requests cooperative cancellation; extraction stops when it reaches an existing cancellation checkpoint.

A token cannot be reset. Once cancellation has been requested, all current and future clones continue to report that request. Create a new token for an unrelated extraction operation.

Cloning is cheap (increments the Arc reference count only).

CandleBackend

Wraps Gliner2Candle behind the NerBackend trait.

Gliner2Candle holds candle tensors which are not Send, so we wrap it in a Mutex to satisfy the Send + Sync requirement of NerBackend.

Methods

detect()

Signature:

pub async fn detect(&self, text: &str, categories: &[EntityCategory]) -> Result<Vec<Entity>, XbergError>

Example:

let result = instance.detect("value", &vec![]).await?;

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text
`categories`	`&\[EntityCategory\]`	Yes	The categories

Returns: Vec<Entity>

Errors: Returns Err(XbergError).

CandleOcrConfig

Legacy common configuration for TrOCR and PaddleOCR-VL.

Use the backend-specific option objects for new code. This type remains for 1.x compatibility and can produce the backend name and common runtime options.

Field	Type	Default	Description
`model`	`CandleModelId`	`CandleModelId::Trocr`	Model (candle model id)
`device`	`DevicePreference`	`Auto`	Device
`cache_dir`	`Option<PathBuf>`	`None`	Optional explicit Hugging Face Hub cache root. When unset, the standard Hugging Face environment-variable conventions are used.
`hf_revision`	`Option<String>`	`None`	Optional immutable Hugging Face revision for caller-selected models.
`max_new_tokens`	`u32`	`4096`	Retained for source compatibility. Current candle OCR engines do not expose per-call token limits; use backend defaults.
`temperature`	`f32`	`0`	Retained for source compatibility. Current candle OCR engines use their model-specific decoding strategy and do not consume this value.

Methods

default()

Signature:

pub fn default() -> CandleOcrConfig

Example:

let result = CandleOcrConfig::default();

Returns: CandleOcrConfig

CaptioningConfig

Since: v1.0

Configuration for the VLM captioning post-processor.

Field	Type	Default	Description
`llm`	`LlmConfig`	—	LLM configuration used for the VLM call.
`prompt`	`Option<String>`	`None`	Optional custom caption prompt. `None` uses the default `RegionKind::Caption` prompt that ships with `crate::llm::region_extractor`.
`min_image_area`	`u32`	`1000`	Skip images whose `width * height` is below this threshold (in pixels). Default `1_000` filters out icons and decorations.
`alt_text`	`CaptionAltTextMode`	`CaptionAltTextMode::Preserve`	Controls whether generated captions preserve, combine with, or replace existing image alternate text.

Methods

default_min_image_area()

Default Self::min_image_area: 1000 px.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_min_image_area() -> u32

Example:

let result = CaptioningConfig::default_min_image_area();

Returns: u32

CaptioningProcessor

Post-processor that captions every extracted image via a VLM.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

CellBBox

A cell bounding box within the reconstructed table grid (pixel coordinates in the crop).

Field	Type	Default	Description
`x1`	`f32`	—	Left edge in crop-pixel coordinates.
`y1`	`f32`	—	Top edge in crop-pixel coordinates.
`x2`	`f32`	—	Right edge in crop-pixel coordinates.
`y2`	`f32`	—	Bottom edge in crop-pixel coordinates.

CellBorders

Per-cell borders (4 sides).

Field	Type	Default	Description
`top`	`Option<BorderStyle>`	`Default::default()`	Top cell border.
`bottom`	`Option<BorderStyle>`	`Default::default()`	Bottom cell border.
`left`	`Option<BorderStyle>`	`Default::default()`	Left (start in LTR) cell border.
`right`	`Option<BorderStyle>`	`Default::default()`	Right (end in LTR) cell border.

CellChange

A single changed cell within a table.

Defined here (rather than only in crate::diff) so RevisionDelta can reference it unconditionally, without requiring the diff Cargo feature. crate::diff re-exports this type verbatim.

Field	Type	Default	Description
`row`	`usize`	—	Zero-based row index.
`col`	`usize`	—	Zero-based column index.
`from`	`String`	—	Value before the change.
`to`	`String`	—	Value after the change.

CellMargins

Cell margins (used for both table-level defaults and per-cell overrides).

Field	Type	Default	Description
`top`	`Option<i32>`	`Default::default()`	Top cell margin in twips.
`bottom`	`Option<i32>`	`Default::default()`	Bottom cell margin in twips.
`left`	`Option<i32>`	`Default::default()`	Left cell margin in twips.
`right`	`Option<i32>`	`Default::default()`	Right cell margin in twips.

CellProperties

Cell-level properties from <w:tcPr>.

Field	Type	Default	Description
`width`	`Option<TableWidth>`	`Default::default()`	Cell width specification.
`grid_span`	`Option<u32>`	`Default::default()`	Number of grid columns this cell spans (default 1).
`v_merge`	`Option<VerticalMerge>`	`Default::default()`	Vertical merge state for this cell.
`borders`	`Option<CellBorders>`	`Default::default()`	Per-cell border overrides.
`shading`	`Option<CellShading>`	`Default::default()`	Cell background shading.
`margins`	`Option<CellMargins>`	`Default::default()`	Per-cell margin overrides.
`vertical_align`	`Option<String>`	`Default::default()`	Vertical text alignment: `"top"`, `"center"`, or `"bottom"`.
`text_direction`	`Option<String>`	`Default::default()`	Text direction: `"lrTb"`, `"tbRl"`, or `"btLr"`.
`no_wrap`	`bool`	—	Whether cell content wraps across lines.

CellShading

Cell shading/background fill.

Field	Type	Default	Description
`fill`	`Option<String>`	`Default::default()`	Background fill color as hex RGB or `"auto"`.
`color`	`Option<String>`	`Default::default()`	Pattern foreground color.
`val`	`Option<String>`	`Default::default()`	Pattern type: `"clear"`, `"solid"`, `"pct10"`, etc.

CharShape

Character formatting attributes from the HWP DocInfo CharShape table.

Field	Type	Default	Description
`bold`	`bool`	—	Bold formatting flag.
`italic`	`bool`	—	Italic formatting flag.
`underline`	`bool`	—	Underline formatting flag.

Chunk

A text chunk with optional embedding and metadata.

Chunks are created when chunking is enabled in ExtractionConfig. Each chunk contains the text content, optional embedding vector (if embedding generation is configured), and metadata about its position in the document.

Field	Type	Default	Description
`content`	`String`	—	The text content of this chunk.
`chunk_type`	`ChunkType`	`ChunkType::Unknown`	Semantic structural classification of this chunk. Assigned by the heuristic classifier based on content patterns and heading context. Defaults to `ChunkType::Unknown` when no rule matches.
`embedding`	`Option<Vec<f32>>`	`None`	Optional embedding vector for this chunk. Only populated when `EmbeddingConfig` is provided in chunking configuration. The dimensionality depends on the chosen embedding model.
`sparse_embedding`	`Option<SparseEmbedding>`	`/* serde(default) */`	Optional sparse (SPLADE) learned embedding for this chunk. Only populated when sparse-embedding generation is configured for chunking. `None` otherwise, including on builds without the `sparse-embeddings` feature. Uses the crate-root `SparseEmbedding` alias rather than `crate::sparse_embeddings::SparseEmbedding` directly: the `sparse_embeddings` module itself only compiles under `sparse-embeddings`/`sparse-embedding-presets`, while the crate-root alias is always defined (a field-compatible stub on builds without either feature), so this field — and `Chunk` itself — compiles on every feature combination, including the crate’s default features.
`late_interaction`	`Option<MultiVectorEmbedding>`	`/* serde(default) */`	Optional ColBERT-style multi-vector (late-interaction) embedding for this chunk. Only populated when late-interaction embedding generation is configured for chunking. `None` otherwise, including on builds without the `late-interaction` feature. Uses the crate-root `MultiVectorEmbedding` alias for the same reason `sparse_embedding` uses `SparseEmbedding` — see that field’s docs.
`metadata`	`ChunkMetadata`	—	Metadata about this chunk’s position and properties.

ChunkClassificationConfig

Since: v1.0

Configuration for the chunk-classification post-processor.

Chunk classification is always multi-label: a chunk may match zero, one, or many of the configured definitions. This is the chunk-level equivalent of PageClassificationConfig, but scoped to individual chunks (ExtractedDocument::chunks) rather than whole pages, and built for large taxonomies where each label needs its own description rather than a bare name.

Field	Type	Default	Description
`prompt_template`	`Option<String>`	`None`	Minijinja prompt template. Receives `{{ definitions }}` (rendered label + description list) and `{{ chunks }}` (a numbered list of chunk texts in the current batch) variables. `None` lets the backend pick a sensible default.
`definitions`	`Vec<ChunkClassificationDefinition>`	—	The set of label definitions the classifier may emit. Must contain at least one entry.
`llm`	`LlmConfig`	—	LLM configuration used for classification.
`batch_size`	`usize`	`10`	Number of chunks batched into a single LLM request. Larger batches amortize the fixed prompt cost (definitions block) across more chunks, at the risk of exceeding the model’s context window for very large taxonomies or chunk texts. Defaults to `DEFAULT_BATCH_SIZE`.
`max_concurrency`	`usize`	`4`	Maximum number of in-flight batch requests. Bounds concurrency against the configured LLM provider. Defaults to `DEFAULT_MAX_CONCURRENCY`.

Methods

default_batch_size()

Default Self::batch_size: DEFAULT_BATCH_SIZE.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_batch_size() -> usize

Example:

let result = ChunkClassificationConfig::default_batch_size();

Returns: usize

default_max_concurrency()

Default Self::max_concurrency: DEFAULT_MAX_CONCURRENCY.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_max_concurrency() -> usize

Example:

let result = ChunkClassificationConfig::default_max_concurrency();

Returns: usize

ChunkClassificationDefinition

Since: v1.0

A single labeled definition the chunk classifier may emit.

Unlike PageClassificationConfig::labels (bare label names), chunk classification targets potentially large domain taxonomies where every label carries its own semantic description, letting the LLM disambiguate similarly named labels without relying on the label string alone.

Field	Type	Default	Description
`label`	`String`	—	Label name returned in `ChunkMetadata::classifications`.
`description`	`String`	—	Semantic description of when this label applies. Injected verbatim into the classification prompt next to the label name.

ChunkClassificationEnrichmentConfig

Chunk-classification enrichment knob: how to multi-label individual chunks.

Operates on ExtractedDocument::chunks in place — the caller must have already produced chunks (e.g. via ExtractionConfig::chunking) for this stage to have any effect; a document with no chunks is a no-op.

Field	Type	Default	Description
`config`	`ChunkClassificationConfig`	—	Label-definition set and LLM/batching settings for the chunk-classification stage.

ChunkClassificationProcessor

Post-processor that asks an LLM to assign multi-label classifications to each chunk of the extracted content.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

ChunkInfo

Information about a single chunk.

Field	Type	Default	Description
`index`	`u32`	—	Zero-based chunk index.
`pages`	`PageRange`	—	Page range for this chunk.
`estimated_time_ms`	`u64`	—	Estimated processing time for this chunk in milliseconds.

ChunkMetadata

Metadata about a chunk’s position in the original document.

Field	Type	Default	Description
`byte_start`	`usize`	—	Byte offset where this chunk starts in the original text (UTF-8 valid boundary).
`byte_end`	`usize`	—	Byte offset where this chunk ends in the original text (UTF-8 valid boundary).
`token_count`	`Option<usize>`	`None`	Number of tokens in this chunk (if available). This is calculated by the embedding model’s tokenizer if embeddings are enabled.
`chunk_index`	`usize`	—	Zero-based index of this chunk in the document.
`total_chunks`	`usize`	—	Total number of chunks in the document.
`first_page`	`Option<u32>`	`None`	First page number this chunk spans (1-indexed). Only populated when page tracking is enabled in extraction configuration.
`last_page`	`Option<u32>`	`None`	Last page number this chunk spans (1-indexed, equal to first_page for single-page chunks). Only populated when page tracking is enabled in extraction configuration.
`heading_context`	`Option<HeadingContext>`	`/* serde(default) */`	Heading context when using Markdown chunker. Contains the heading hierarchy this chunk falls under. Only populated when `ChunkerType::Markdown` is used.
`heading_path`	`Vec<String>`	`/* serde(default) */`	Flattened heading trail from document root to this chunk’s section. Each element is a heading’s text, outermost first. Derived from `heading_context` when present; empty otherwise. Provides a binding-friendly, RAG-shaped breadcrumb without requiring callers to walk the nested `HeadingContext` structure.
`image_indices`	`Vec<u32>`	`/* serde(default) */`	Indices into `ExtractedDocument.images` for images on pages covered by this chunk. Contains zero-based indices into the top-level `images` collection for every image whose `page_number` falls within `[first_page, last_page]`. Empty when image extraction is disabled or the chunk spans no pages with images.
`node_ids`	`Vec<String>`	`/* serde(default) */`	Ids of the `DocumentNode`s this chunk was derived from. Joins a chunk back to the structured document tree via `DocumentNode::id`. Populated from exact node provenance when available, with a textual containment fallback for rendered chunks that do not retain byte offsets.
`page_spans`	`Vec<PageSpan>`	`/* serde(default) */`	Per-page bounding-box spans this chunk covers, for viewer highlighting (#1295). One entry per page the chunk overlaps, in page order — the first and last entries’ `page` fields equal `first_page`/`last_page`. Populated whenever page-boundary provenance is available (the same condition under which `first_page`/`last_page` are populated); each entry’s `bbox` is additionally populated when the document’s structured node tree (`ExtractedDocument::document`) is available, as the union of that page’s body-layer node bounding boxes found within this chunk. Empty when page-boundary provenance is unavailable (mirrors `first_page`/ `last_page` being `None`).
`classifications`	`Vec<ClassificationLabel>`	`/* serde(default) */`	Multi-label classification result for this chunk. Populated by the chunk-classification post-processor when `ExtractionConfig::chunk_classification` is set. A chunk may match zero, one, or many of the configured label definitions. Empty when chunk classification was not configured.

ChunkingConfig

Chunking configuration.

Configures text chunking for document content, including chunk size, overlap, trimming behavior, and optional embeddings.

Use ..the default constructor when constructing to allow for future field additions:

Field	Type	Default	Description
`max_characters`	`usize`	`1000`	Maximum size per chunk (in units determined by `sizing`). When `sizing` is `Characters` (default), this is the max character count. When using token-based sizing, this is the max token count. Default: 1000
`overlap`	`usize`	`200`	Overlap between chunks (in units determined by `sizing`). Default: 200
`trim`	`bool`	`true`	Whether to trim whitespace from chunk boundaries. Default: true
`chunker_type`	`ChunkerType`	`ChunkerType::Text`	Type of chunker to use (Text or Markdown). Default: Text
`embedding`	`Option<EmbeddingConfig>`	`None`	Optional embedding configuration for chunk embeddings.
`sparse_embedding`	`Option<SparseEmbeddingConfig>`	`None`	Optional sparse (SPLADE) embedding configuration for chunk embeddings. When set, sparse vectors are generated for each chunk’s content and attached via `sparse_embedding`. Requires the `sparse-embeddings` feature; without it, a warning is emitted and no sparse vectors are attached. Config-file only: like `RerankerConfig` and the local-ONNX branch of `embedding`, this has no CLI flag and no environment variable. Only the secret/identity fields of LLM-routed configs (model, API key, base URL) get that reach.
`late_interaction`	`Option<LateInteractionConfig>`	`None`	Optional late-interaction (ColBERT) embedding configuration for chunk embeddings. When set, multi-vector embeddings are generated for each chunk’s content and attached via `late_interaction`. Requires the `late-interaction` feature; without it, a warning is emitted and no late-interaction vectors are attached. Config-file only, for the same reason as `sparse_embedding` above.
`preset`	`Option<String>`	`None`	Use a preset configuration (overrides individual settings if provided).
`sizing`	`ChunkSizing`	`ChunkSizing::Characters`	How to measure chunk size. Default: `Characters` (Unicode character count). Enable `chunking-tiktoken` or `chunking-tokenizers` features for token-based sizing.
`topic_threshold`	`Option<f32>`	`None`	Optional cosine similarity threshold for semantic topic boundary detection. Only used when `chunker_type` is `Semantic` and an `EmbeddingConfig` is provided. You almost never need to set this. When omitted, defaults to `0.75` which works well for most documents. Lower values detect more topic boundaries (more, smaller chunks); higher values detect fewer. Range: `0.0..=1.0`.
`table_chunking`	`TableChunkingMode`	`TableChunkingMode::Split`	How to handle markdown tables that exceed the chunk size limit. Only applies when `chunker_type` is `Markdown`. - `Split` (default) — tables are split at row boundaries; continuation chunks do not repeat the header. - `RepeatHeader` — the table header row and separator are prepended to every continuation chunk so each chunk is self-contained. Default: `Split`

Methods

default()

Signature:

pub fn default() -> ChunkingConfig

Example:

let result = ChunkingConfig::default();

Returns: ChunkingConfig

ChunkingProcessor

Post-processor that chunks text in document content.

This processor:

Runs in the Middle processing stage
Only processes when config.chunking is configured
Stores chunks in result.chunks
Uses configurable chunk size and overlap

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

estimated_duration_ms()

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

Citation

A structured citation from a citation block.

Parsed from entries like: [^srcN]: source, locator, excerpt: "text"

Field	Type	Default	Description
`label`	`String`	—	The label of the citation (e.g., “src1” in `[^src1]: ...`).
`source`	`String`	—	The source reference (path, URL, or identifier).
`locator`	`Option<String>`	`None`	Optional locator within the source (e.g., “page 3” or “section 2.1”).
`excerpt`	`Option<String>`	`None`	Optional excerpt — quoted text from the source.

CitationExtractor

Citation format extractor for RIS, PubMed/MEDLINE, and EndNote XML formats.

Parses citation files and extracts structured bibliography data including entries, authors, publication years, and format-specific metadata.

Methods

default()

Signature:

pub fn default() -> CitationExtractor

Example:

let result = CitationExtractor::default();

Returns: CitationExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

CitationMetadata

Citation file metadata (RIS, PubMed, EndNote).

Field	Type	Default	Description
`citation_count`	`usize`	—	Total number of citation records in the file.
`format`	`Option<String>`	`Default::default()`	Detected citation file format (e.g. `"ris"`, `"pubmed"`, `"endnote"`).
`authors`	`Vec<String>`	`vec![]`	Author names collected across all citation records.
`year_range`	`Option<YearRange>`	`Default::default()`	Earliest and latest publication years found in the file.
`dois`	`Vec<String>`	`vec![]`	DOI identifiers found in the citation records.
`keywords`	`Vec<String>`	`vec![]`	Keywords collected from all citation records.

ClassicOcrCacheManifest

Stable, status-free result of seeding the complete classical OCR catalog.

Field	Type	Default	Description
`schema_version`	`u32`	—	Manifest schema version.
`xberg_version`	`String`	—	Xberg version that defined the catalog.
`cache_dir`	`PathBuf`	—	Explicit Hugging Face cache root populated by the operation.
`catalog_count`	`usize`	—	Number of distinct backend/catalog identifier pairs.
`artifact_count`	`usize`	—	Number of artifacts in `Self::models`.
`total_size_bytes`	`u64`	—	Sum of verified artifact sizes.
`models`	`Vec<ClassicOcrModelEntry>`	—	Complete catalog, sorted by stable model coordinates.

ClassicOcrModelEntry

One checksummed model artifact in the classical OCR cache.

Field	Type	Default	Description
`backend`	`String`	—	OCR backend that consumes the artifact.
`catalog_id`	`String`	—	Stable logical model identifier shared by companion artifacts.
`role`	`String`	—	Model role within that backend.
`artifact_kind`	`String`	—	Artifact kind, either `model` or `dictionary`.
`repo`	`String`	—	Hugging Face repository containing the artifact.
`revision`	`String`	—	Registry revision used to resolve the artifact.
`file`	`String`	—	Repository-relative artifact path.
`sha256`	`String`	—	Expected lowercase SHA-256 digest.
`license`	`String`	—	SPDX license identifier for the model weights.
`size_bytes`	`u64`	—	Verified artifact size in bytes.
`path`	`PathBuf`	—	Resulting local path used by the runtime.

ClassificationLabel

A single label + confidence pair.

Field	Type	Default	Description
`label`	`String`	—	Label name as configured in `PageClassificationConfig::labels`.
`confidence`	`Option<f32>`	`None`	Backend-reported confidence in `[0.0, 1.0]`. `None` when the backend (e.g. an LLM prompt without explicit confidence schema) did not report one.

CleanOutcome

Result of clean_obsolete: how many stray files were removed and how many removals failed (e.g. permissions).

Field	Type	Default	Description
`removed`	`usize`	—	Files successfully deleted.
`failed`	`usize`	—	Files that could not be deleted.

CodeChunkInfo

A single structurally-meaningful code chunk produced by tree-sitter parsing.

Purpose-built payload owned by xberg — deliberately does not expose the upstream tree_sitter_language_pack types, so binding generators never need to resolve an external crate’s types across FFI/language boundaries.

Field	Type	Default	Description
`text`	`String`	—	The raw source text of this chunk.
`context_path`	`Vec<String>`	—	Hierarchical path of enclosing structural items (e.g. `["MyClass", "my_method"]`).
`node_types`	`Vec<String>`	—	Tree-sitter node kinds that appear at the top level of this chunk (e.g. `"function_definition"`, `"class_definition"`).
`byte_start`	`usize`	—	Inclusive start byte offset of this chunk in the original source.
`byte_end`	`usize`	—	Exclusive end byte offset of this chunk in the original source.

CodeDataAttribute

An XML-style attribute attached to an Element node.

Populated only for CodeDataNodeKind::Element; always empty for KeyValue and Sequence nodes.

Field	Type	Default	Description
`name`	`String`	—	Attribute name (e.g. `"class"`, `"href"`).
`value`	`String`	—	Attribute value as a raw string (quotes stripped).
`byte_start`	`usize`	—	Inclusive start byte offset of the `name="value"` attribute token.
`byte_end`	`usize`	—	Exclusive end byte offset of the `name="value"` attribute token.

CodeDataNode

A node in the hierarchical data tree produced by data-format extraction.

Purpose-built payload owned by xberg — mirrors tree_sitter_language_pack::DataNode but flattens its Span down to plain byte offsets, so binding generators never need to resolve an external crate’s types across FFI/language boundaries.

Field	Type	Default	Description
`kind`	`CodeDataNodeKind`	—	Whether this node is a key/value pair, XML element, or sequence item.
`key`	`Option<String>`	`/* serde(default) */`	Key, attribute name, tag name, or positional index (`"0"`, `"1"`, …). `None` at the document root.
`value`	`Option<String>`	`/* serde(default) */`	Leaf scalar value, if any. `None` for containers (objects, arrays, XML elements with child elements).
`attributes`	`Vec<CodeDataAttribute>`	`/* serde(default) */`	Attributes on element-shape nodes (XML `STag` attributes). Empty for all other kinds.
`children`	`Vec<CodeDataNode>`	`/* serde(default) */`	Children for nested containers and XML element bodies.
`byte_start`	`usize`	—	Inclusive start byte offset of this node in the original source.
`byte_end`	`usize`	—	Exclusive end byte offset of this node in the original source.

CodeExtractor

Source code extractor using tree-sitter language pack.

Detects the programming language from the file extension or shebang line, then uses tree-sitter to parse and extract structural information.

Methods

default()

Signature:

pub fn default() -> CodeExtractor

Example:

let result = CodeExtractor::default();

Returns: CodeExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

CodeMetadata

Code-format metadata: the structural chunks produced by tree-sitter parsing.

Wrapped by FormatMetadata::Code. Kept as a named struct (rather than an inline enum-variant body) so serde can tag it under internal tagging and utoipa can emit a referenceable CodeMetadata component in the OpenAPI schema.

Field	Type	Default	Description
`chunks`	`Vec<CodeChunkInfo>`	`vec![]`	Structural code chunks (function/class/module boundaries).
`data`	`Option<CodeDataNode>`	`Default::default()`	Hierarchical key/value data tree extracted from data-format source (JSON, YAML, TOML, XML, CSV, etc.), when data extraction was enabled.

ColorScheme

Color scheme containing all 12 standard Office theme colors.

Field	Type	Default	Description
`name`	`String`	—	Color scheme name.
`dk1`	`Option<ThemeColor>`	`Default::default()`	Dark 1 (dark background) color.
`lt1`	`Option<ThemeColor>`	`Default::default()`	Light 1 (light background) color.
`dk2`	`Option<ThemeColor>`	`Default::default()`	Dark 2 color.
`lt2`	`Option<ThemeColor>`	`Default::default()`	Light 2 color.
`accent1`	`Option<ThemeColor>`	`Default::default()`	Accent color 1.
`accent2`	`Option<ThemeColor>`	`Default::default()`	Accent color 2.
`accent3`	`Option<ThemeColor>`	`Default::default()`	Accent color 3.
`accent4`	`Option<ThemeColor>`	`Default::default()`	Accent color 4.
`accent5`	`Option<ThemeColor>`	`Default::default()`	Accent color 5.
`accent6`	`Option<ThemeColor>`	`Default::default()`	Accent color 6.
`hlink`	`Option<ThemeColor>`	`Default::default()`	Hyperlink color.
`fol_hlink`	`Option<ThemeColor>`	`Default::default()`	Followed hyperlink color.

ColumnLayout

Column layout configuration.

Field	Type	Default	Description
`count`	`Option<i32>`	`Default::default()`	Number of columns.
`space_twips`	`Option<i32>`	`Default::default()`	Space between columns in twips.
`equal_width`	`Option<bool>`	`Default::default()`	Whether columns have equal width; if `false`, individual widths may vary.

Comment

A comment from word/comments.xml (#82).

Joined to the body text via w:commentRangeStart/w:commentRangeEnd (anchors, not carrying content) and w:commentReference (the marker matched to this id).

Field	Type	Default	Description
`id`	`String`	—	Comment identifier matching `w:commentReference/@w:id`.
`author`	`Option<String>`	`Default::default()`	Comment author, if present.
`paragraphs`	`Vec<Paragraph>`	`vec![]`	Paragraphs of comment content.

CommonPdfMetadata

Common PDF metadata fields extracted from the document info dictionary.

Used as an intermediate type during extraction before building PdfExtractionMetadata.

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Document title from the PDF Info dictionary.
`subject`	`Option<String>`	`Default::default()`	Document subject from the PDF Info dictionary.
`authors`	`Option<Vec<String>>`	`vec![]`	Document authors parsed from the PDF Author field.
`keywords`	`Option<Vec<String>>`	`vec![]`	Keywords parsed from the PDF Keywords field.
`created_at`	`Option<String>`	`Default::default()`	Creation timestamp in ISO 8601 format.
`modified_at`	`Option<String>`	`Default::default()`	Last modification timestamp in ISO 8601 format.
`created_by`	`Option<String>`	`Default::default()`	Creator application or author name.

ConcurrencyConfig

Controls thread usage for constrained environments.

Set max_threads to cap all internal thread pools (Rayon, ONNX Runtime intra-op), batch concurrency and Tesseract recognition to a single limit. Set max_concurrent_ocr to give recognition a limit of its own, which is the knob to reach for when the host has cores to spare but not the memory to run a recognition session on each of them. It is applied as given and is not capped by max_threads. The first extraction in a process fixes it for that process — see the field’s own documentation.

Default budget when `max_threads` is unset

Without an explicit max_threads, the effective budget is min(detected_cpu_cores, 8) — a deliberate ceiling chosen for serverless/shared-tenant defaults, not a full-host auto-scale. On a host with more than 8 cores this means the extra cores go unused unless one of the following applies:

max_threads is set explicitly above 8 (the only way to exceed the ceiling on a bare-metal or VM host with no CPU quota).
The process runs under a Linux cgroup CPU quota (containers, Kubernetes resources.limits.cpu); in that case the quota itself is used as the ceiling instead of the hardcoded 8, since the quota already reflects a deliberately-configured resource limit.

When neither applies and the host has more than 8 cores, a single WARN-level log is emitted the first time the budget is resolved, naming the detected core count and the applied cap, so the ceiling is discoverable without reading source.

Field	Type	Default	Description
`max_threads`	`Option<usize>`	`Default::default()`	Maximum number of threads for all internal thread pools. Caps Rayon global pool size, ONNX Runtime intra-op threads, and the combined document/inner-task budget for batch extraction. When `None`, the effective budget is `min(detected_cpu_cores, 8)` unless a Linux cgroup CPU quota is present, in which case the quota is used as the ceiling instead. On hosts with more than 8 cores and no cgroup quota, set `max_threads` explicitly to use the additional cores — the default will not scale past 8 on its own.
`max_concurrent_ocr`	`Option<usize>`	`Default::default()`	Maximum number of Tesseract recognition sessions that run at once. When `None`, recognition follows `max_threads`, reduced to the number of sessions the host’s free memory holds. Each session keeps its own page image and recognition working set resident, so a host with many cores and little memory needs this lower than the thread budget. Set it to `4` to keep the fixed limit that releases up to 1.2.6 applied. A value set here is applied as given: neither the thread budget nor the memory reading reduces it. Both of those bound the automatic limit, and a caller who names a number has already decided what the host can carry. The first extraction in a process fixes the limit for the rest of that process, and a later extraction that names a different value keeps the first one. The two limiters that enforce it — the admission semaphore in the Tesseract backend and the handle pool behind it — are built once inside a backend the plugin registry holds for the life of the process, and the pool’s capacity is fixed when it is constructed. A later value could therefore be reported but never enforced. Set it on the first extraction, or run one process per value.

ContentConfig

Content extraction and conversion configuration.

Controls how HTML is converted to the output format. Uses html-to-markdown-rs as the conversion engine for all formats (markdown, plain text, djot).

Field	Type	Default	Description
`output_format`	`String`	`"markdown"`	Output format: `"markdown"` (default), `"plain"`, `"djot"`.
`preprocessing_preset`	`String`	`"standard"`	Preprocessing aggressiveness: `"minimal"`, `"standard"` (default), `"aggressive"`. - Minimal: only scripts/styles removed. - Standard: also removes nav, nav-hinted headers/footers/asides, forms. - Aggressive: removes all footers/asides unconditionally.
`remove_navigation`	`bool`	`true`	Remove navigation elements (nav, breadcrumbs, menus). Default: `true`.
`remove_forms`	`bool`	`true`	Remove form elements. Default: `true`.
`strip_tags`	`Vec<String>`	`vec![]`	HTML tag names to strip (render children only, remove the tag wrapper). Default: `[]`.
`preserve_tags`	`Vec<String>`	`vec![]`	HTML tag names to preserve as raw HTML in output.
`exclude_selectors`	`Vec<String>`	`["noscript"]`	CSS selectors for elements to exclude entirely (element + all content). Unlike `strip_tags` (which removes the wrapper but keeps children), excluded elements and all descendants are dropped. Supports CSS selectors: `.class`, `#id`, `[attribute]`, compound selectors. Default: `["noscript"]`. `<noscript>` fallback content (no-JS notices, tracking pixels, GTM iframes) is meant for browsers with JavaScript disabled, not for a markdown reader, and `strip_tags` cannot drop it — on `preprocessing_preset: "standard"` (crawlberg’s only path) it only removes the wrapper and still renders the children. Example: `[".cookie-banner", "#ad-container", "[role='complementary']"]`
`skip_images`	`bool`	`false`	Skip image elements in output. Default: `false`.
`max_depth`	`Option<usize>`	`None`	Max DOM traversal depth. Prevents stack overflow on deeply nested HTML.
`wrap`	`bool`	`false`	Enable line wrapping. Default: `false`.
`wrap_width`	`usize`	`80`	Wrap width when `wrap` is enabled. Default: `80`.
`include_document_structure`	`bool`	`true`	Include document structure tree in output. Default: `true`.
`extract_metadata`	`bool`	`true`	Prepend a YAML frontmatter block (`title`, `description`, etc., extracted from `<head>`) to the markdown output. Default: `true`. This only controls the frontmatter text inside `markdown.content`. Crawlberg never reads `<head>` metadata back out of the converter’s result – `PageMetadata` is populated independently by `crate::html::metadata::extract_metadata` from the parsed DOM, so turning this off does not lose any metadata field.

ContentFilterConfig

Cross-extractor content filtering configuration.

Controls whether “furniture” content (headers, footers, page numbers, watermarks, repeating text) is included in or stripped from extraction results. Applies across all extractors (PDF, DOCX, RTF, ODT, HTML, etc.) with format-specific implementation.

When None on ExtractionConfig, each extractor uses its current default behavior unchanged.

Field	Type	Default	Description
`include_headers`	`bool`	`false`	Include running headers in extraction output. - PDF: Disables top-margin furniture stripping and prevents the layout model from treating `PageHeader`-classified regions as furniture. - DOC/DOCX: Includes document headers in text output. - RTF/ODT: Headers already included; this is a no-op when true. - HTML/EPUB: Keeps `<header>` element content. Default: `false` (headers are stripped or excluded).
`include_footers`	`bool`	`false`	Include running footers in extraction output. - PDF: Disables bottom-margin furniture stripping and prevents the layout model from treating `PageFooter`-classified regions as furniture. - DOC/DOCX: Includes document footers in text output. - RTF/ODT: Footers already included; this is a no-op when true. - HTML/EPUB: Keeps `<footer>` element content. Default: `false` (footers are stripped or excluded).
`include_footnotes`	`bool`	`false`	Include footnote bodies in extraction output. - PDF: Prevents the layout model from treating `Footnote`-classified regions as furniture, so footnote bodies survive alongside the main text instead of being silently dropped. - DOC: Includes footnote definitions in extracted output. - Other formats: No effect currently. Default: `false` (footnotes are stripped), matching the existing `include_headers` / `include_footers` defaults.
`strip_repeating_text`	`bool`	`true`	Enable the heuristic cross-page repeating text detector. When `true` (default), text that repeats verbatim across a supermajority of pages is classified as furniture and stripped. Disable this if brand names or repeated headings are being incorrectly removed by the heuristic. This flag also gates a same-page rule: a body paragraph is removed when its text is also carried by a table detected on the same page (catches table content that PDF extraction renders both as a table and as body text). The comparison preserves case and never touches headings, list items, code blocks, formulas, or captions, so it cannot delete a body sentence merely for repeating an earlier heading’s words (GH#1623). Note: when a layout-detection model is active, the model may independently classify page-header / page-footer / footnote regions as furniture on a per-page basis. To preserve those regions, set `include_headers = true`, `include_footers = true`, `include_footnotes = true`, or any combination, in addition to disabling this flag. Primarily affects PDF extraction. Default: `true`.
`include_watermarks`	`bool`	`false`	Include watermark text in extraction output. - PDF: Keeps watermark artifacts and arXiv identifiers. - Other formats: No effect currently. Default: `false` (watermarks are stripped).

Methods

default()

Signature:

pub fn default() -> ContentFilterConfig

Example:

let result = ContentFilterConfig::default();

Returns: ContentFilterConfig

ContributorRole

JATS contributor with role.

Field	Type	Default	Description
`name`	`String`	—	Contributor display name.
`role`	`Option<String>`	`None`	Contributor role (e.g. `"author"`, `"editor"`).

ConversionOptions

Main conversion options for HTML to Markdown conversion.

Use ConversionOptions::builder() to construct, or the default constructor for defaults.

Field	Type	Default	Description
`heading_style`	`HeadingStyle`	`HeadingStyle::Atx`	Heading style to use in Markdown output (ATX `#` or Setext underline).
`list_indent_type`	`ListIndentType`	`ListIndentType::Spaces`	How to indent nested list items (spaces or tab).
`list_indent_width`	`usize`	`2`	Number of spaces (or tabs) to use for each level of list indentation.
`bullets`	`String`	`"-*+"`	Bullet character(s) to use for unordered list items (e.g. `"-"`, `"*"`).
`strong_em_symbol`	`String`	`"*"`	Character used for bold/italic emphasis markers (`*` or `_`).
`escape_asterisks`	`bool`	`false`	Escape `*` characters in plain text to avoid unintended bold/italic.
`escape_underscores`	`bool`	`false`	Escape `_` characters in plain text to avoid unintended bold/italic.
`escape_misc`	`bool`	`false`	Escape miscellaneous Markdown metacharacters (`[]()#` etc.) in plain text.
`escape_ascii`	`bool`	`false`	Escape ASCII characters that have special meaning in certain Markdown dialects.
`code_language`	`String`	`""`	Default language annotation for fenced code blocks that have no language hint.
`autolinks`	`bool`	`true`	Automatically convert bare URLs into Markdown autolinks.
`default_title`	`bool`	`false`	Emit a default title when no `<title>` tag is present.
`br_in_tables`	`bool`	`false`	Render `<br>` elements inside table cells as literal line breaks.
`compact_tables`	`bool`	`false`	Emit tables without column padding (compact GFM format). When `true`, column widths are not computed and cells are emitted with no trailing spaces. Separator rows use exactly `---` per column. Produces token-efficient output suitable for RAG / LLM contexts. Default `false` (aligned padding preserved).
`highlight_style`	`HighlightStyle`	`HighlightStyle::DoubleEqual`	Style used for `<mark>` / highlighted text (e.g. `==text==`).
`extract_metadata`	`bool`	`true`	Populate `result.metadata` with `<head>` / `<meta>` extraction (title, description, Open Graph, Twitter Card, JSON-LD, …). Default `true`. Disabling skips the metadata pass only — table extraction into `result.tables` runs unconditionally.
`whitespace_mode`	`WhitespaceMode`	`WhitespaceMode::Normalized`	Controls how whitespace sequences are normalised in the converted output. - `WhitespaceMode::Normalized` (default) — collapses consecutive whitespace characters (spaces, tabs, newlines) to a single space, matching browser rendering behaviour. - `WhitespaceMode::Strict` — preserves all whitespace exactly as it appears in the source HTML, including runs of spaces and embedded newlines. Choose `Strict` only when the source HTML uses deliberate whitespace (e.g. pre-formatted content outside `<pre>` tags). For most documents `Normalized` produces cleaner output.
`strip_newlines`	`bool`	`false`	Strip all newlines from the output, producing a single-line result.
`wrap`	`bool`	`false`	Wrap long lines at `wrap_width` characters.
`wrap_width`	`usize`	`80`	Maximum output line width in characters when `wrap` is `true` (default `80`). Lines are broken at word boundaries so that no line exceeds this length. A value of `0` is treated as “no limit” — equivalent to leaving `wrap` disabled. Has no effect when `wrap` is `false`.
`convert_as_inline`	`bool`	`false`	Treat the entire document as inline content (no block-level wrappers).
`sub_symbol`	`String`	`""`	Markdown notation for subscript text (e.g. `"~"`).
`sup_symbol`	`String`	`""`	Markdown notation for superscript text (e.g. `"^"`).
`newline_style`	`NewlineStyle`	`NewlineStyle::Spaces`	How to encode hard line breaks (`<br>`) in Markdown.
`code_block_style`	`CodeBlockStyle`	`CodeBlockStyle::Backticks`	Style used for fenced code blocks (backticks or tilde).
`keep_inline_images_in`	`Vec<String>`	`vec![]`	HTML tag names whose `<img>` children are kept inline instead of block.
`preprocessing`	`PreprocessingOptions`	—	Options for the HTML pre-processing pass applied before conversion begins. Pre-processing runs before the HTML is handed to the converter and can perform operations such as unwrapping redundant wrapper elements, removing tracking pixels, and normalising vendor-specific markup. See `PreprocessingOptions` for the full set of knobs. Defaults to `PreprocessingOptions::default()`, which enables the standard cleaning passes. Set individual fields on `PreprocessingOptions` (or construct via `ConversionOptions::builder`) to opt in or out of specific passes.
`encoding`	`String`	`"utf-8"`	Expected character encoding of the input HTML (default `"utf-8"`).
`debug`	`bool`	`false`	Emit debug information during conversion.
`strip_tags`	`Vec<String>`	`vec![]`	HTML tag names whose content is stripped from the output entirely.
`preserve_tags`	`Vec<String>`	`vec![]`	HTML tag names that are preserved verbatim in the output.
`skip_images`	`bool`	`false`	Skip conversion of `<img>` elements (omit images from output).
`inline_data_media`	`InlineDataMedia`	`InlineDataMedia::Keep`	What the output shows for an image or embedded media element whose address is an inline `data:` URL: the URL with its payload (`InlineDataMedia::Keep`, the default), the alt text alone, or nothing. See `InlineDataMedia` for the elements it covers.
`url_escape_style`	`UrlEscapeStyle`	`UrlEscapeStyle::Angle`	URL encoding strategy for link and image destinations. Controls how special characters in URL destinations are escaped: - `UrlEscapeStyle::Angle` (default) — wraps the destination in angle brackets when it contains spaces or newlines. Some parsers misinterpret `>` inside such a destination. - `UrlEscapeStyle::Percent` — percent-encodes every character that is not an RFC 3986 unreserved character or `/`, producing a destination that all Markdown parsers handle correctly even when the URL contains `<`, `>`, spaces, or parentheses.
`link_style`	`LinkStyle`	`LinkStyle::Inline`	Link rendering style (inline or reference).
`output_format`	`HtmlToMarkdownRsOutputFormat`	`HtmlToMarkdownRsOutputFormat::Markdown`	Target output format (Markdown, plain text, etc.).
`include_document_structure`	`bool`	`false`	Include structured document tree in result.
`extract_images`	`bool`	`false`	Extract inline images from data URIs and SVGs.
`max_image_size`	`u64`	`5242880`	Maximum decoded image size in bytes (default 5MB).
`max_input_size`	`Option<u64>`	`None`	Maximum accepted HTML input size in bytes. WebAssembly builds default to `DEFAULT_WASM_MAX_INPUT_SIZE` (2 MiB) to avoid an uncatchable stack-exhaustion trap on unusually large DOMs. Native builds default to `None`. Set `Some(bytes)` to choose another limit or `None` to disable it.
`capture_svg`	`bool`	`false`	Capture SVG elements as images.
`infer_dimensions`	`bool`	`true`	Infer image dimensions from data.
`max_depth`	`Option<usize>`	`None`	Maximum DOM traversal depth. `None` uses the library’s internal native-stack safety limit. Explicit values above that safety limit are clamped to prevent process-aborting stack overflows on pathologically deep DOM trees.
`exclude_selectors`	`Vec<String>`	`vec![]`	CSS selectors for elements to exclude entirely (element + all content). Unlike `strip_tags` (which removes the tag wrapper but keeps children), excluded elements and all their descendants are dropped from the output. Supports any CSS selector that `tl` supports: tag names, `.class`, `#id`, `[attribute]`, etc. Invalid selectors are silently skipped at conversion time. Example: `vec![".cookie-banner".into(), "#ad-container".into(), "[role='complementary']".into()]`
`tier_strategy`	`TierStrategy`	`TierStrategy::Auto`	Which conversion tier to use. - `TierStrategy::Auto` (default) — automatically choose the best path. - `TierStrategy::Tier2` — always use the Tier-2 DOM-walk path. - `TierStrategy::Tier1` — always attempt Tier-1 (testkit only).
`base_url`	`Option<String>`	`None`	Base URL to resolve relative `href`/`src` destinations against. When set, every relative link and image/media destination (`a href`, `img src` and its lazy-load fallbacks, `srcset`, `graphic` `url`/`href`/`xlink:href`/`src`, `iframe`/`audio`/`video`/`source` `src`) is resolved to an absolute URL before being written to the Markdown output, so the result is followable without the reader knowing where the source HTML came from. A `<base href>` in the document, if present, is honored the way a browser honors it: it is itself resolved against `base_url`, and that combined result becomes the effective base every other relative reference resolves against. Non-hierarchical schemes (`mailto:`, `tel:`, `javascript:`, `data:`) and already-absolute URLs are left unchanged. An unset (default) or unparseable `base_url`, and a relative reference that fails to resolve, leave the original attribute text unchanged – this option never panics and never corrupts a destination it cannot confidently resolve. Default `None` — no resolution, output byte-identical to versions before this option existed.

CoreProperties

Dublin Core metadata from docProps/core.xml

Contains standard metadata fields defined by the Dublin Core standard and Office-specific extensions.

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Document title
`subject`	`Option<String>`	`Default::default()`	Document subject/topic
`creator`	`Option<String>`	`Default::default()`	Document creator/author
`keywords`	`Option<String>`	`Default::default()`	Keywords or tags
`description`	`Option<String>`	`Default::default()`	Document description/abstract
`last_modified_by`	`Option<String>`	`Default::default()`	User who last modified the document
`revision`	`Option<String>`	`Default::default()`	Revision number
`created`	`Option<String>`	`Default::default()`	Creation timestamp (ISO 8601)
`modified`	`Option<String>`	`Default::default()`	Last modification timestamp (ISO 8601)
`category`	`Option<String>`	`Default::default()`	Document category
`content_status`	`Option<String>`	`Default::default()`	Content status (Draft, Final, etc.)
`language`	`Option<String>`	`Default::default()`	Document language
`identifier`	`Option<String>`	`Default::default()`	Unique identifier
`version`	`Option<String>`	`Default::default()`	Document version
`last_printed`	`Option<String>`	`Default::default()`	Last print timestamp (ISO 8601)

CrawlConfig

Configuration for crawl, scrape, and map operations.

Field	Type	Default	Description
`max_depth`	`Option<usize>`	`None`	Maximum crawl depth (number of link hops from the start URL).
`max_pages`	`Option<usize>`	`None`	Maximum number of pages to crawl.
`max_links_per_page`	`Option<usize>`	`None`	Maximum links enqueued from a single page. Defaults to 10000. Bounds the work one hostile or pathological page can create; links past the cap are dropped and a warning is logged.
`max_concurrent`	`Option<usize>`	`None`	Maximum number of concurrent requests.
`crawl_strategy`	`CrawlStrategyKind`	`CrawlStrategyKind::Bfs`	Traversal order. Defaults to breadth-first. A frontier or strategy set explicitly on `CrawlEngineBuilder` takes precedence over this field.
`content_filter`	`Option<ContentFilterKind>`	`None`	Content filter applied to each page. `None` keeps every page. A content filter set explicitly on `CrawlEngineBuilder` takes precedence.
`bm25_query`	`Option<String>`	`None`	Query the BM25 content filter scores pages against. Required by `ContentFilterKind::Bm25`.
`bm25_threshold`	`Option<f64>`	`None`	Minimum BM25 score a page must reach to be kept. Defaults to `0.0`.
`respect_robots_txt`	`bool`	`false`	Whether to respect robots.txt directives. A crawl that respects them also honours the page’s own robots instructions: it does not follow the links of a page marked `nofollow` by its robots meta tag or an `X-Robots-Tag` header. A link marked `rel="nofollow"` is a hint, not a robots directive, and is still followed. A `noindex` page is still crawled and its links followed; the page result marks it with `noindex_detected`.
`soft_http_errors`	`bool`	`false`	When true, HTTP-level error responses (404 NotFound, 403 Forbidden, WAF blocks) are surfaced as `ScrapeResult` records with the matching `status_code` rather than raised as `CrawlError`. A WAF block reports the status of the refused response when it is a 4xx or 5xx, and 403 when the block page came with a 2xx status. Default `false` preserves the historical throw-on-error contract for direct fetches. Independently of this flag, 404s reached at the end of a redirect chain are always surfaced softly — the user opted into redirect-following, so receiving a 404 there is part of the normal flow rather than an unexpected error.
`user_agent`	`Option<String>`	`None`	Custom user-agent string.
`stay_on_domain`	`bool`	`false`	Whether to confine document links (`.pdf`, `.docx`, `.zip`, …) to the seed domain. Page links are always confined to the seed host, widened to its subdomains by `Self::allow_subdomains`; this flag does not loosen that. It applies only to document links, which are classified by file extension before their host is considered and so are followed cross-host by default – the usual case being documents served from a CDN or object store. Set this to `true` to require documents to live on the seed domain too.
`allow_subdomains`	`bool`	`false`	Whether subdomains of the seed host are in scope. Applies to page links unconditionally, and to document links when `Self::stay_on_domain` is set.
`include_paths`	`Vec<String>`	`vec![]`	Regex patterns for paths to include during crawling.
`exclude_paths`	`Vec<String>`	`vec![]`	Regex patterns for paths to exclude during crawling.
`path_patterns_match_query`	`bool`	`false`	Whether `include_paths`/`exclude_paths` match against `path?query` instead of just `path`. Defaults to `false`, matching path only: a pattern anchored with `$` (e.g. `/feed/?$`) changes meaning once the query joins the matched text, so this must stay opt-in rather than silently changing what an existing config matches. Has no effect when `Self::path_patterns_match_url` is `true`.
`path_patterns_match_url`	`bool`	`false`	Whether `include_paths`/`exclude_paths` match against the full URL, `scheme://host[:port]/path?query`, so a pattern can scope by host. Defaults to `false`. When `true` it takes precedence over `Self::path_patterns_match_query`: the query is part of the full URL whatever that flag says. The matched text never contains a `user:password@`, a fragment or a default port, and the host is in punycode (`bücher.de` is matched as `xn--bcher-kva.de`).
`dedup_include_query`	`bool`	`false`	Whether the crawl-dedup key includes the (sorted) query string. Defaults to `false`, matching historical behavior: `/item?id=1` and `/item?id=2` are treated as one page and only the first is fetched. `true` keeps the query, sorted, in the key, so each distinct query is fetched once.
`strip_tracking_params`	`bool`	`false`	Whether to strip `tracking_params` from a discovered URL before it is deduplicated, fetched, and reported. Defaults to `false`, so no tracking parameters are stripped unless explicitly enabled.
`tracking_params`	`Vec<String>`	`["utm_*", "fbclid", "gclid", "ref"]`	Query parameter name patterns to strip when `strip_tracking_params` is `true`. A pattern ending in `` matches by prefix (`utm_` matches `utm_source`, `utm_campaign`, …); any other pattern matches the parameter name exactly. Defaults to `["utm_*", "fbclid", "gclid", "ref"]`, applied only once `strip_tracking_params` is enabled.
`custom_headers`	`HashMap<String, String>`	`HashMap::new()`	Custom HTTP headers to send with each request to the seed URL’s host. A request to another host does not carry them.
`request_timeout`	`std::time::Duration`	`30000ms`	Timeout for individual HTTP requests (in milliseconds when serialized).
`rate_limit_ms`	`Option<u64>`	`None`	Per-domain rate limit in milliseconds. When set, enforces a minimum delay between requests to the same domain. Defaults to 200ms when `None`.
`max_redirects`	`usize`	`10`	Maximum number of redirects to follow.
`retry_count`	`usize`	`0`	Number of retry attempts for failed requests. Bounded by `MAX_RETRY_COUNT`.
`retry_codes`	`Vec<u16>`	`vec![]`	HTTP status codes that should trigger a retry. When empty, every rate limit, server error, bad gateway and timeout is retried. When set, only a failure whose status is listed is retried, so a timeout without a response is not.
`retry_initial_delay_ms`	`u64`	`100`	Initial delay, in milliseconds, before the first retry. Doubled on each subsequent attempt (capped at `retry_max_delay_ms`). Defaults to 100ms.
`retry_max_delay_ms`	`u64`	`60000`	Upper bound, in milliseconds, on the exponential retry backoff. Defaults to 60s.
`rate_limit_jitter_ratio`	`f64`	`0`	Fraction of the per-domain rate-limit delay to randomly jitter by, in `[0.0, 1.0]`. `0.0` (the default) applies no jitter and preserves the previous fixed-interval behaviour; `0.1` jitters the delay by up to ±10%.
`cookies_enabled`	`bool`	`false`	Whether to enable cookie handling.
`auth`	`Option<AuthConfig>`	`None`	Authentication configuration.
`max_body_size`	`Option<usize>`	`None`	Maximum response body size in bytes. `None` does not mean unbounded: an unset cap falls back to a 100 MiB safety ceiling, because HTTP responses are decompressed while being read and a few hundred compressed bytes can otherwise expand to gigabytes in memory. To read bodies larger than that, set this explicitly.
`remove_tags`	`Vec<String>`	`vec![]`	CSS selectors for tags to remove from HTML before processing.
`content`	`ContentConfig`	—	Content extraction and conversion configuration.
`map_limit`	`Option<usize>`	`None`	Maximum number of URLs to return from a map operation.
`map_search`	`Option<String>`	`None`	Search filter for map results (case-insensitive substring match on URLs).
`download_assets`	`bool`	`false`	Whether to download assets (CSS, JS, images, etc.) from the page.
`asset_types`	`Vec<AssetCategory>`	`vec![]`	Filter for asset categories to download.
`max_asset_size`	`Option<usize>`	`None`	Maximum size in bytes for individual asset downloads.
`browser`	`BrowserConfig`	—	Browser configuration.
`proxy`	`Option<ProxyConfig>`	`None`	Proxy configuration for HTTP requests.
`user_agents`	`Vec<String>`	`vec![]`	List of user-agent strings for rotation. If non-empty, overrides `user_agent`.
`capture_screenshot`	`bool`	`false`	Whether to capture a screenshot when using the browser. Only supported by `scrape()` with `BrowserBackend::Chromiumoxide` and `BrowserMode::Always` or `Stealth`. A screenshot is 100–500 KB of PNG per page, so `crawl()` does not carry screenshots in `CrawlPageResult`/`CrawlResult` at all — a multi-thousand-page crawl holding one per page in memory is not a safe default. Setting this with any other configuration (a different backend, `BrowserMode::Auto`/`Never`, or during `crawl()`) has no effect and logs a warning rather than silently doing nothing.
`follow_document_urls`	`bool`	`false`	Re-enqueue discovered `LinkType::Document` URLs into the crawl frontier so the crawl follows links from document pages (PDFs, etc.) as it would from HTML pages. Default: `false` (documents terminate at materialisation).
`document_url_depth`	`Option<u32>`	`None`	Maximum document-depth (from the seed URL through document links only) when `follow_document_urls` is true. `None` means inherit `max_depth`. Independent of `max_depth`: a document URL is enqueued only if BOTH the outer `max_depth` and (if set) `document_url_depth` permit it.
`download_documents`	`bool`	`true`	Whether to download non-HTML documents (PDF, DOCX, images, code, etc.) instead of skipping them. Defaults to `true` — unlike `download_assets` and `capture_screenshot`, which default to `false`.
`document_max_size`	`Option<usize>`	`52428800`	Maximum size in bytes for document downloads. Defaults to 50 MB.
`document_mime_types`	`Vec<String>`	`vec![]`	Allowlist of MIME types to download. If empty, uses built-in defaults.
`document_output_dir`	`Option<PathBuf>`	`None`	Directory to stream downloaded document bytes into instead of holding them in memory on `DownloadedDocument.content`. When set, `content` is left empty and `DownloadedDocument.content_path` is populated with `<dir>/<content_hash>.<ext>`. `None` (default) preserves today’s in-memory-only behavior. Has no effect on wasm32, which has no filesystem — use `document_content_encoding` there instead.
`document_content_encoding`	`Option<DocumentContentEncoding>`	`None`	Opt-in encoding that duplicates `DownloadedDocument.content` into a serializable field for language bindings that need the bytes in-memory (`content` itself is `alef(skip)`ed). `None` (default) means no encoding is produced. Independent of `document_output_dir` — set both to get a file on disk and an in-memory copy.
`warc_output`	`Option<PathBuf>`	`None`	Path to write WARC output. If `None`, WARC output is disabled.
`browser_profile`	`Option<String>`	`None`	Named browser profile for persistent sessions (cookies, localStorage). Chromiumoxide backend only. The native backend runs an in-process JavaScript engine with no Chrome process and therefore no profile directory, so this is ignored there and logs a warning. It is also ignored — with a warning — when a shared browser pool is in use (the pool launches before any per-crawl config exists) or when connecting to an external CDP endpoint whose process crawlberg does not own.
`save_browser_profile`	`bool`	`false`	Whether to save changes back to the browser profile on exit.
`ssrf`	`SsrfPolicy`	`crawlberg::SsrfPolicy::from_env()`	SSRF policy for outbound network requests. Default: deny private networks, allow http/https only, max 5 redirects. All policy fields are exposed to language bindings. wasm32 (including Node.js): `deny_private` does not stop hostname-based requests. There is no DNS resolution on this target, so only a literal IP host is checked against the policy — a domain name is always permitted, regardless of `deny_private`. Under Node, where `fetch` enforces no CORS, this means a service embedding the wasm binding can be driven to internal hosts by domain name even with `deny_private = true`. Enforce egress restrictions at the network layer for that deployment target; do not rely on this field. See `crawlberg::net::validate_url`.
`ssrf_deny_private_explicit`	`Option<bool>`	`None`	Pins `SsrfPolicy::deny_private` to a caller-chosen value, bypassing the `CRAWLBERG_ALLOW_PRIVATE_NETWORK` operator override entirely for this config. `ssrf.deny_private` is a plain, always-serialized `bool`: several alef-generated bindings construct `SsrfPolicy::default()` (hardcoding `deny_private: true`) whenever their caller never touches SSRF settings at all, so `true` on that field alone cannot distinguish “the caller wants private networks denied” from “the binding’s own structural default landed on `true`”. The environment variable exists precisely to resolve that ambiguity in the common case by treating any `true` as inconclusive and deferring to the operator. Set this field when that default-deferral is wrong for your call — e.g. a test that must prove `deny_private: true` still denies even while the operator has set `CRAWLBERG_ALLOW_PRIVATE_NETWORK` suite-wide for every other call. `None` (default) preserves today’s behavior: the environment variable may still flip `ssrf.deny_private` to `false`. `Some(value)` pins `ssrf.deny_private` to `value` and the environment variable is not consulted for this config.
`dispatch`	`Option<DispatchProfile>`	`None`	Pluggable dispatch components: bypass provider, escalation strategy, retry policy, WAF classifier, domain state, escalation budget, and max_total_attempts. When `None`, the engine uses its built-in defaults (no bypass, `BrowserOnly` strategy, `SimpleRetryPolicy`, built-in WAF classifier, no domain state, unlimited budget, 10 total attempt cap). Rust-only advanced field. Generated language bindings do not expose pluggable dispatch components; language clients use the built-in dispatch defaults configured by the Rust engine. Not serializable — Rust callers construct this at runtime and skip it in TOML/JSON configs.
`credential_scope`	`Option<CredentialScope>`	`None`	The seed host that credentials are scoped to, and the credentials the seed URL carried. Set by the engine when it admits a seed URL; a caller cannot build one and leaves it `None`.
`proxy_provider`	`Option<ProxyProvider>`	`None`	Optional `ProxyProvider` for per-request proxy rotation. Takes precedence over the static `ProxyConfig` in `proxy` when set. A native browser render goes through the proxy it picks for the page, unless `browser.proxy` is set. The Chrome backend refuses to render with a provider unless `browser.proxy` is set. Not serializable — Rust callers inject at runtime.

CsvConfig

Since: v1.1

Configuration for CSV/TSV extraction.

When unset (ExtractionConfig::csv == None), the extractor keeps its existing default behavior: the delimiter is auto-detected by sampling the file (comma, tab, pipe, or semicolon), and no line is treated as a comment.

Field	Type	Default	Description
`delimiter`	`Option<String>`	`Default::default()`	Field delimiter, as a single-character string (e.g. `","`, `";"`, `"\t"`, `"\|"`). When `None` (default), the delimiter is auto-detected from a sample of the file. Must be exactly one ASCII byte when set — `ExtractionConfig::validate` rejects an empty string or a multi-byte value with a helpful error. The TSV MIME type (`text/tab-separated-values`) always forces `\t` regardless of this setting.
`comment_prefixes`	`Vec<String>`	`vec![]`	Line prefixes that mark a comment line to skip entirely during row parsing (e.g. `["#"]`). A line is treated as a comment when its trimmed start matches any of these prefixes exactly. Default: empty, meaning no line is treated as a comment (matches the pre-existing extractor behavior).

CsvExtractor

CSV/TSV extractor with proper field parsing.

Replaces raw text passthrough with structured CSV parsing, producing space-separated text output and populated tables field.

Methods

default()

Signature:

pub fn default() -> CsvExtractor

Example:

let result = CsvExtractor::default();

Returns: CsvExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

CsvMetadata

CSV/TSV file metadata.

Field	Type	Default	Description
`row_count`	`u32`	—	Total number of data rows (excluding the header row if present).
`column_count`	`u32`	—	Number of columns detected.
`delimiter`	`Option<String>`	`Default::default()`	Field delimiter character (e.g. `","` or `"\t"`).
`has_header`	`bool`	—	Whether the first row was treated as a header.
`column_types`	`Option<Vec<String>>`	`vec![]`	Inferred data type for each column (e.g. `"string"`, `"integer"`, `"float"`).

CustomProperties

Custom properties from docProps/custom.xml

Maps property names to their values. Values are converted to JSON types based on the VT (Variant Type) specified in the XML.

DbfExtractor

Extractor for dBASE (.dbf) files.

Reads all records and formats them as a markdown table with column headers derived from field names.

Methods

default()

Signature:

pub fn default() -> DbfExtractor

Example:

let result = DbfExtractor::default();

Returns: DbfExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Reads a .dbf file from disk and, when a .dbt/.fpt memo sidecar sits next to it, resolves Memo fields to their full text instead of failing on MissingMemoFile (#109). Falls back to memo-less parsing when there is no sidecar, matching extract_content’s behavior.

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

DbfFieldInfo

dBASE field information.

Field	Type	Default	Description
`name`	`String`	—	Field (column) name.
`field_type`	`String`	—	dBASE field type character (e.g. `"C"` for character, `"N"` for numeric).

DbfMetadata

dBASE (DBF) file metadata.

Field	Type	Default	Description
`record_count`	`usize`	—	Total number of data records in the DBF file.
`field_count`	`usize`	—	Number of field (column) definitions.
`fields`	`Vec<DbfFieldInfo>`	`vec![]`	Descriptor for each field in the table schema.

DeepseekOcrBackend

DeepSeek-OCR backend using candle transformers.

A vision-language model combining SAM vision encoder, ViT/Qwen2 vision transformer, CLIP projection, and language decoder for multimodal OCR.

Configuration

DeepSeek-OCR accepts backend options for weight source, device, version, and dtype:

{
  "device": "auto",
  "model_id": "deepseek-ai/DeepSeek-OCR",
  "version": 2,
  "dtype": "auto"
}

device (string): "auto" (default), "cpu", "cuda", "metal"
model_id (string): HuggingFace repo id to auto-download weights from. Defaults to deepseek-ai/DeepSeek-OCR, pinned to a checksum-verified revision. Ignored when model_path is set.
model_path (string, optional): path to a local model directory. Takes precedence over model_id. When omitted, the weights named by model_id are downloaded on first use into the standard Hugging Face cache – no manual staging required.
hf_revision (string, optional): immutable commit for a custom model_id. The default model is pinned automatically.
cache_dir (string, optional): explicit Hugging Face Hub cache root. When omitted, HF_HUB_CACHE, HUGGINGFACE_HUB_CACHE, and HF_HOME are honored.
version (integer): model version 1 or 2 (default: 2)
dtype (string): "auto" (default, picks BF16 on CUDA / F16 on Metal / F32 on CPU), "f32", "f16", or "bf16". A dtype with no kernel on the selected device fails the load hard rather than silently falling back – this is not a tuning knob.

Methods

default()

Signature:

pub fn default() -> DeepseekOcrBackend

Example:

let result = DeepseekOcrBackend::default();

Returns: DeepseekOcrBackend

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Process an image using the DeepSeek-OCR engine.

Errors:

Returns Validation if image_bytes is empty. Returns Ocr if weight download, device selection, engine initialisation, or inference fails.

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Process an image file using the DeepSeek-OCR engine.

Errors:

Returns an error if the file cannot be read or if inference fails.

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

emits_structured_markdown()

Signature:

pub fn emits_structured_markdown(&self) -> bool

Example:

let result = instance.emits_structured_markdown();

Returns: bool

confidence_semantics()

DeepSeek-OCR reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

DeepseekOcrBackendOptions

Runtime options accepted by the candle-deepseek-ocr backend.

Field	Type	Default	Description
`model_path`	`Option<String>`	`Default::default()`	Local DeepSeek-OCR model directory. Takes precedence over `model_id` when present.
`model_id`	`Option<String>`	`Default::default()`	Optional Hugging Face repository identifier. Defaults to the checksum-pinned `deepseek-ai/DeepSeek-OCR`. Ignored when `model_path` is set.
`hf_revision`	`Option<String>`	`Default::default()`	Optional immutable Hugging Face model revision. The default model is pinned automatically; a custom `model_id` requires this to be set explicitly.
`cache_dir`	`Option<String>`	`Default::default()`	Optional Hugging Face cache root.
`device`	`Option<CandleDevicePreference>`	`Default::default()`	Optional per-call device override.
`version`	`Option<u32>`	`Default::default()`	DeepSeek-OCR model generation, either 1 or 2. Defaults to 2.
`dtype`	`Option<CandleDeepseekOcrDtype>`	`Default::default()`	Optional weight precision override; see `CandleDeepseekOcrDtype`.

DetectMimeTypeParams

Request parameters for MIME type detection.

Field	Type	Default	Description
`path`	`String`	—	Path to the file
`use_content`	`bool`	`true`	Use content-based detection (default: true)

DetectResponse

MIME type detection response.

Field	Type	Default	Description
`mime_type`	`String`	—	Detected MIME type
`filename`	`Option<String>`	`None`	Original filename (if provided)

DetectTimings

Granular timing breakdown for a single detect() call.

Field	Type	Default	Description
`preprocess_ms`	`f64`	—	Time spent in image preprocessing (resize, letterbox, normalize, tensor allocation).
`onnx_ms`	`f64`	—	Time for the ONNX `session.run()` call (actual neural network computation).
`model_total_ms`	`f64`	—	Total time from start of model call to end of raw output decoding.
`postprocess_ms`	`f64`	—	Time spent in postprocessing heuristics (confidence filtering, overlap resolution).

DetectedBoundary

A detected structural boundary in the text.

Field	Type	Default	Description
`byte_offset`	`usize`	—	Byte offset of the start of the line in the original text.
`is_header`	`bool`	—	Whether this boundary looks like a header/section title.

DetectionResult

Page-level detection result containing all detections and page metadata.

Field	Type	Default	Description
`page_width`	`u32`	—	Page width in pixels (as seen by the model).
`page_height`	`u32`	—	Page height in pixels (as seen by the model).
`detections`	`Vec<LayoutDetection>`	—	All layout detections on this page after postprocessing.

DiagramEdge

A diagram edge: one open connector joining two nodes.

Field	Type	Default	Description
`from`	`usize`	—	Index into `DiagramGraph::nodes` of the connector’s start.
`to`	`usize`	—	Index into `DiagramGraph::nodes` of the connector’s end.
`bidirectional`	`bool`	—	Whether the connector is drawn with an arrowhead at both ends. A single arrowhead sets `from` and `to`, so it needs no flag. When the source draws no arrowhead at all, the connector’s own point order stands in for direction and this stays `false`.
`label`	`Option<String>`	`None`	Text drawn on the connector, `None` when it carries none.
`stroke`	`Option<String>`	`None`	Stroke colour as `#rrggbb`.
`dashed`	`bool`	—	Whether the connector is dashed.

DiagramGraph

A graph recovered from one vector diagram.

Field	Type	Default	Description
`name`	`Option<String>`	`Default::default()`	Diagram name, taken from the source’s own title where it has one.
`nodes`	`Vec<DiagramNode>`	`vec![]`	Nodes in reading order (top to bottom, then left to right).
`edges`	`Vec<DiagramEdge>`	`vec![]`	Edges, ordered by `(from, to)`.

DiagramNode

A diagram node: one closed outline plus the text drawn inside it.

Field	Type	Default	Description
`id`	`String`	—	Stable identifier, `n0`, `n1`, … assigned in reading order.
`label`	`String`	—	Text found inside the outline, joined with `\n` in reading order. Empty when the shape carries no text.
`shape`	`DiagramShape`	—	Outline shape.
`fill`	`Option<String>`	`None`	Fill colour as `#rrggbb`, `None` when the shape is unfilled or filled with a gradient or pattern rather than a flat colour.
`stroke`	`Option<String>`	`None`	Stroke colour as `#rrggbb`, under the same restriction as `fill`.
`stroke_width`	`Option<f32>`	`None`	Stroke width in source units.
`dashed`	`bool`	—	Whether the outline is dashed.

DiffHunk

A single contiguous hunk in a unified diff.

Field	Type	Default	Description
`from_line`	`usize`	—	Starting line number in the old content (0-indexed).
`from_count`	`usize`	—	Number of lines from the old content in this hunk.
`to_line`	`usize`	—	Starting line number in the new content (0-indexed).
`to_count`	`usize`	—	Number of lines from the new content in this hunk.
`lines`	`Vec<DiffLine>`	—	Lines that make up this hunk.

DiffOptions

Options controlling how two ExtractedDocument values are compared.

Field	Type	Default	Description
`include_metadata`	`bool`	`true`	Include metadata changes in the diff. Default: `true`.
`include_embedded`	`bool`	`true`	Include embedded-children changes in the diff. Default: `true`.
`max_content_chars`	`Option<usize>`	`None`	Truncate content to this many characters before diffing. Useful for very large documents where only the first N characters matter. `None` means no truncation.

Methods

default()

Signature:

pub fn default() -> DiffOptions

Example:

let result = DiffOptions::default();

Returns: DiffOptions

DjotAttributeGroup

Attributes associated with a named Djot element.

Field	Type	Default	Description
`identifier`	`String`	—	Element identifier used by the Djot attribute map.
`attributes`	`Attributes`	—	Attributes associated with the element.

DjotContent

Comprehensive Djot document structure with semantic preservation.

This type captures the full richness of Djot markup, including:

Block-level structures (headings, lists, blockquotes, code blocks, etc.)
Inline formatting (emphasis, strong, highlight, subscript, superscript, etc.)
Attributes (classes, IDs, key-value pairs)
Links, images, footnotes
Math expressions (inline and display)
Tables with full structure

Available when the djot feature is enabled.

Field	Type	Default	Description
`plain_text`	`String`	—	Plain text representation for backwards compatibility
`blocks`	`Vec<FormattedBlock>`	—	Structured block-level content
`metadata`	`Metadata`	—	Metadata from YAML frontmatter
`tables`	`Vec<Table>`	—	Extracted tables as structured data
`images`	`Vec<DjotImage>`	—	Extracted images with metadata
`links`	`Vec<DjotLink>`	—	Extracted links with URLs
`footnotes`	`Vec<Footnote>`	—	Footnote definitions
`attributes`	`Vec<DjotAttributeGroup>`	`/* serde(default) */`	Attributes mapped by element identifier (if present)

DjotExtractor

Djot markup extractor with metadata and table support.

Parses Djot documents with YAML frontmatter, extracting:

Metadata from YAML frontmatter
Plain text content
Tables as structured data
Document structure (headings, links, code blocks)

Methods

default()

Signature:

pub fn default() -> DjotExtractor

Example:

let result = DjotExtractor::default();

Returns: DjotExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

DjotImage

Image element in Djot.

Field	Type	Default	Description
`src`	`String`	—	Image source URL or path
`alt`	`String`	—	Alternative text
`title`	`Option<String>`	`None`	Optional title
`attributes`	`Option<Attributes>`	`None`	Element attributes

DjotLink

Link element in Djot.

Field	Type	Default	Description
`url`	`String`	—	Link URL
`text`	`String`	—	Link text content
`title`	`Option<String>`	`None`	Optional title
`attributes`	`Option<Attributes>`	`None`	Element attributes

DocExtractor

Native DOC extractor using OLE/CFB parsing.

This extractor handles Word 97-2003 binary (.doc) files without requiring LibreOffice, providing ~50x faster extraction.

Methods

default()

Signature:

pub fn default() -> DocExtractor

Example:

let result = DocExtractor::default();

Returns: DocExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

DocOrientationDetector

Detects document page orientation using the PP-LCNet model.

Thread-safe: the model runs behind &self through the inference seam, which owns the session synchronization. On native targets the model is downloaded from HuggingFace on first use and cached locally; on wasm32 it is constructed from bytes supplied by the caller (see Self::from_bytes).

DocProperties

Document properties from <wp:docPr>.

Field	Type	Default	Description
`id`	`Option<String>`	`Default::default()`	Unique numeric identifier for this drawing within the document.
`name`	`Option<String>`	`Default::default()`	Human-readable name for this drawing object.
`description`	`Option<String>`	`Default::default()`	Alt-text description for accessibility.

DocTagsExtractor

Extractor for Docling DocTags documents.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

DocbookExtractor

DocBook document extractor.

Supports both DocBook 4.x (no namespace) and 5.x (with namespace) formats.

Methods

default()

Signature:

pub fn default() -> DocbookExtractor

Example:

let result = DocbookExtractor::default();

Returns: DocbookExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

DoclingCompatDocument

Document content in the docling-serve response format.

Field	Type	Default	Description
`md_content`	`String`	—	Markdown content of the converted document

DoclingCompatResponse

OpenWebUI “Docling” engine response format.

Returned by POST /v1/convert/file for docling-serve compatibility.

Field	Type	Default	Description
`document`	`DoclingCompatDocument`	—	Converted document content
`status`	`String`	—	Processing status

DoctorCheck

A single doctor verdict: what was checked, the outcome, and why.

Field	Type	Default	Description
`name`	`String`	—	Check identifier, e.g. `ocr.tesseract` or `layout.rtdetr`.
`status`	`ProbeStatus`	—	Pass / warn / fail / skip verdict.
`message`	`String`	—	One-line reason or detail (e.g. missing language, resolved path, error).

Methods

pass()

A ProbeStatus::Pass verdict: the checked backend or setting will work as configured on this host.

Signature:

pub fn pass(name: String, message: String) -> DoctorCheck

Example:

let result = DoctorCheck::pass("value", "value");

Parameters:

Name	Type	Required	Description
`name`	`String`	Yes	The name
`message`	`String`	Yes	The message

Returns: DoctorCheck

warn()

A ProbeStatus::Warn verdict: the check ran and found something actionable, but nothing is broken. Never fails the report.

Signature:

pub fn warn(name: String, message: String) -> DoctorCheck

Example:

let result = DoctorCheck::warn("value", "value");

Parameters:

Name	Type	Required	Description
`name`	`String`	Yes	The name
`message`	`String`	Yes	The message

Returns: DoctorCheck

fail()

A ProbeStatus::Fail verdict: the configured setup will not work (or will silently degrade) on this host.

Signature:

pub fn fail(name: String, message: String) -> DoctorCheck

Example:

let result = DoctorCheck::fail("value", "value");

Parameters:

Name	Type	Required	Description
`name`	`String`	Yes	The name
`message`	`String`	Yes	The message

Returns: DoctorCheck

skip()

A ProbeStatus::Skip verdict: the check cannot run locally; first real use decides, possibly after a download.

Signature:

pub fn skip(name: String, message: String) -> DoctorCheck

Example:

let result = DoctorCheck::skip("value", "value");

Parameters:

Name	Type	Required	Description
`name`	`String`	Yes	The name
`message`	`String`	Yes	The message

Returns: DoctorCheck

DoctorReport

Aggregate doctor report over all configured backends and settings.

Field	Type	Default	Description
`checks`	`Vec<DoctorCheck>`	`vec![]`	Individual check verdicts, in execution order.

Methods

is_ok()

Whether no check failed. Warnings and skips do not count as failures, so the report stays usable as a scripting/CI gate.

Signature:

pub fn is_ok(&self) -> bool

Example:

let result = instance.is_ok();

Returns: bool

DocumentBoundary

Detected document boundary within a PDF.

Field	Type	Default	Description
`start_page`	`u32`	—	1-indexed start page (inclusive).
`end_page`	`u32`	—	1-indexed end page (inclusive).
`confidence`	`f32`	—	Confidence in this boundary, `[0.0, 1.0]`.
`reason`	`BoundaryReason`	—	Reason for the boundary detection.

DocumentCounts

Cheap structural counts for an extracted document.

Populated on every ExtractedDocument returned by extract / extract_batch, regardless of whether the heavy pages / images collections are materialized. A caller that only needs “how many pages / tables / images did this document have?” (reporting, cost estimation, progress, quotas) can read these without enabling per-page or per-image extraction.

The page count comes from the parse (the extractor already walks the page tree); it does not require opting into per-page content. pages is 0 for inputs that are not page-addressable (e.g. plain text).

Field	Type	Default	Description
`pages`	`usize`	—	Total pages in the source document (`0` when not page-addressable).
`tables`	`usize`	—	Tables detected in the document.
`images`	`usize`	—	Images detected in the document.

DocumentExtractor

Trait for document extractor plugins.

Implement this trait to add support for new document formats or override built-in extraction behavior. Foreign-language bindings expose the DocumentExtractor::extract method, which accepts ExtractInput and returns an ExtractedDocument.

Priority System

When multiple extractors support the same MIME type, the registry selects the extractor with the highest priority value. Use this to:

Override built-in extractors (priority > 50)
Provide fallback extractors (priority < 50)
Implement specialized extractors for specific use cases

Default priority is 50.

Thread Safety

Extractors must be thread-safe (Send + Sync) to support concurrent extraction.

Methods

extract()

Binding-safe extraction entry point for foreign-language plugin bridges.

Accepts the same unified input shape as the public extraction API and returns one extracted document result.

Signature:

pub async fn extract(&self, input: ExtractInput, config: &ExtractionConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.extract(todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`input`	`ExtractInput`	Yes	The input data
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Get the list of MIME types supported by this extractor.

Can include exact MIME types and prefix patterns:

Exact: "application/pdf", "text/plain"
Prefix: "image/*" (matches any image type)

Returns:

A slice of MIME type strings.

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Get the priority of this extractor.

Higher priority extractors are preferred when multiple extractors support the same MIME type.

Priority Guidelines

0-25: Fallback/low-quality extractors
26-49: Alternative extractors
50: Default priority (built-in extractors)
51-75: Premium/enhanced extractors
76-100: Specialized/high-priority extractors

Returns:

Priority value (default: 50)

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

can_handle()

Optional: Check if this extractor can handle a specific file.

Allows for more sophisticated detection beyond MIME types. Defaults to true (rely on MIME type matching).

Returns:

true if the extractor can handle this file, false otherwise.

Signature:

pub fn can_handle(&self, path: &PathBuf, mime_type: &str) -> bool

Example:

let result = instance.can_handle("value", "value");

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	The path
`mime_type`	`&str`	Yes	The mime type

Returns: bool

DocumentExtractorRegistry

Registry for document extractor plugins.

Manages extractors with MIME type and priority-based selection.

Thread Safety

The registry is thread-safe and can be accessed concurrently from multiple threads.

Methods

default()

Signature:

pub fn default() -> DocumentExtractorRegistry

Example:

let result = DocumentExtractorRegistry::default();

Returns: DocumentExtractorRegistry

DocumentMetadata

Metadata about a document for analysis.

Field	Type	Default	Description
`mime_type`	`String`	—	MIME type of the document.
`size_bytes`	`u64`	—	File size in bytes.
`page_count`	`Option<u32>`	`None`	Page count (if known, e.g., from previous analysis).
`force_ocr`	`bool`	—	Whether OCR is forced regardless of text layer.
`user_chunk_config`	`Option<UserChunkConfig>`	`None`	User-provided chunk configuration overrides.
`chunking_enabled`	`bool`	—	Whether chunking is enabled for this job.

DocumentNode

A single node in the document tree.

Each node has deterministic id, typed content, optional parent/children for tree structure, and metadata like page number, bounding box, and content layer.

Field	Type	Default	Description
`id`	`String`	`/* serde(default) */`	Deterministic identifier (hash of node type + text + page + position). Stable and unique within a single extraction response: every internal construction path threads the node’s position (its index in `DocumentStructure::nodes`) into the hash, so identical `(node_type, text, page)` tuples at different positions never collide. Always serialised — `ChunkMetadata::node_ids` references it to join chunks back to the nodes they were derived from. `#[serde(default)]` covers the missing-field case on inbound JSON (e.g. documents serialised before this field existed).
`content`	`NodeContent`	—	Node content — tagged enum, type-specific data only.
`parent`	`Option<u32>`	`None`	Parent node index (`None` = root-level node).
`children`	`Vec<u32>`	`/* serde(default) */`	Child node indices in reading order.
`content_layer`	`ContentLayer`	`ContentLayer::Body`	Content layer classification. Always serialised — Kotlin-Android (and any other typed binding) treats the field as non-nullable, so omitting it from the JSON wire would break consumer deserialisation. `#[serde(default)]` covers the missing-field case on inbound JSON.
`page`	`Option<u32>`	`None`	Page number where this node starts (1-indexed).
`page_end`	`Option<u32>`	`None`	Page number where this node ends (for multi-page tables/sections).
`bbox`	`Option<BoundingBox>`	`None`	Bounding box in document coordinates.
`annotations`	`Vec<TextAnnotation>`	`/* serde(default) */`	Inline annotations (formatting, links) on this node’s text content. Only meaningful for text-carrying nodes; empty for containers.
`attributes`	`Option<HashMap<String, String>>`	`None`	Format-specific key-value attributes. Extensible bag for miscellaneous data without a dedicated typed field: CSS classes, LaTeX environment names, Excel cell formulas, slide layout names, etc.

DocumentRelationship

A resolved relationship between two nodes in the document tree.

Field	Type	Default	Description
`source`	`u32`	—	Source node index (the referencing node).
`target`	`u32`	—	Target node index (the referenced node).
`kind`	`RelationshipKind`	—	Semantic kind of the relationship.

DocumentRevision

A single tracked change embedded in a document.

Populated by per-format extractors that understand change-tracking metadata (DOCX w:ins/w:del/w:rPrChange, ODT text:change-*, …). Every extractor defaults to ExtractedDocument.revisions = None until a format-specific implementation is added.

Field	Type	Default	Description
`revision_id`	`String`	—	Format-specific revision identifier. For DOCX this is the `w:id` attribute value on the change element (e.g. `"42"`). When the attribute is absent a synthetic fallback is generated (`"docx-ins-0"`, `"docx-del-3"`, …).
`author`	`Option<String>`	`None`	Display name of the author who made this change, when available.
`timestamp`	`Option<String>`	`None`	ISO-8601 timestamp of the change, when available. Stored as a plain string so this type remains FFI-friendly and unconditionally available without the `chrono` optional dep. DOCX populates this from the `w:date` attribute (e.g. `"2024-03-15T10:30:00Z"`).
`kind`	`RevisionKind`	—	Semantic kind of this revision.
`anchor`	`Option<RevisionAnchor>`	`None`	Best-effort document location for this revision. Resolution is format-dependent and may be `None` when the location cannot be determined (e.g. changes inside table cells before table-cell anchor support is added).
`delta`	`RevisionDelta`	—	The content changes that make up this revision.

DocumentStructure

Top-level structured document representation.

A flat array of nodes with index-based parent/child references forming a tree. Root-level nodes have parent: None. Use body_roots() and furniture_roots() to iterate over top-level content by layer.

Validation

Call validate() after construction to verify all node indices are in bounds and parent-child relationships are bidirectionally consistent.

Field	Type	Default	Description
`nodes`	`Vec<DocumentNode>`	`vec![]`	All nodes in document/reading order.
`source_format`	`Option<String>`	`None`	Origin format identifier (e.g. “docx”, “pptx”, “html”, “pdf”). Allows renderers to apply format-aware heuristics when converting the document tree to output formats.
`relationships`	`Vec<DocumentRelationship>`	`vec![]`	Resolved relationships between nodes (footnote refs, citations, anchor links, etc.). Populated during derivation from the internal document representation. Empty when no relationships are detected.
`node_types`	`Vec<String>`	`vec![]`	Sorted, deduplicated list of node type names present in this document. Each value is the snake_case `node_type` tag of the corresponding `NodeContent` variant (e.g. `"paragraph"`, `"heading"`, `"table"`, …). Computed from `nodes` via `DocumentStructure::finalize_node_types`. Empty until that method is called (internal construction paths call it at the end of derivation).

Methods

finalize_node_types()

Compute and populate the node_types field from the current nodes.

Call this after all nodes have been added to the structure. Internal construction paths (builder, derivation) call this automatically.

Signature:

pub fn finalize_node_types(&mut self)

Example:

use xberg::types::document_structure::{DocumentStructure, DocumentNode, NodeContent};

let mut structure = DocumentStructure {
    nodes: vec![DocumentNode {
        id: String::new(),
        content: NodeContent::Paragraph { text: "Hello".into() },
        parent: None,
        children: vec![],
        content_layer: Default::default(),
        page: None,
        page_end: None,
        bbox: None,
        annotations: vec![],
        attributes: None,
    }],
    source_format: None,
    relationships: vec![],
    node_types: vec![],
};
structure.finalize_node_types();
assert!(structure.node_types.contains(&"paragraph".to_string()));

Returns: No return value.

is_empty()

Check if the document structure is empty.

Signature:

pub fn is_empty(&self) -> bool

Example:

let result = instance.is_empty();

Returns: bool

default()

Signature:

pub fn default() -> DocumentStructure

Example:

let result = DocumentStructure::default();

Returns: DocumentStructure

DocumentSummary

Summary of an extracted document.

Field	Type	Default	Description
`text`	`String`	—	Summary text (plain prose).
`strategy`	`SummaryStrategy`	—	Strategy that produced this summary.
`token_count`	`Option<u32>`	`None`	Approximate token count of the summary, when known.

DocxAppProperties

Application properties from docProps/app.xml for DOCX

Contains Word-specific document statistics and metadata.

Field	Type	Default	Description
`application`	`Option<String>`	`Default::default()`	Application name (e.g., “Microsoft Office Word”)
`app_version`	`Option<String>`	`Default::default()`	Application version
`template`	`Option<String>`	`Default::default()`	Template filename
`total_time`	`Option<i32>`	`Default::default()`	Total editing time in minutes
`pages`	`Option<i32>`	`Default::default()`	Number of pages
`words`	`Option<i32>`	`Default::default()`	Number of words
`characters`	`Option<i32>`	`Default::default()`	Number of characters (excluding spaces)
`characters_with_spaces`	`Option<i32>`	`Default::default()`	Number of characters (including spaces)
`lines`	`Option<i32>`	`Default::default()`	Number of lines
`paragraphs`	`Option<i32>`	`Default::default()`	Number of paragraphs
`company`	`Option<String>`	`Default::default()`	Company name
`doc_security`	`Option<i32>`	`Default::default()`	Document security level
`scale_crop`	`Option<bool>`	`Default::default()`	Scale crop flag
`links_up_to_date`	`Option<bool>`	`Default::default()`	Links up to date flag
`shared_doc`	`Option<bool>`	`Default::default()`	Shared document flag
`hyperlinks_changed`	`Option<bool>`	`Default::default()`	Hyperlinks changed flag

DocxExtractor

High-performance DOCX extractor.

This extractor provides:

Fast text extraction via streaming XML parsing
Comprehensive metadata extraction (core.xml, app.xml, custom.xml)

Methods

default()

Signature:

pub fn default() -> DocxExtractor

Example:

let result = DocxExtractor::default();

Returns: DocxExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

DocxMetadata

Word document metadata.

Extracted from DOCX files using shared Office Open XML metadata extraction. Integrates with office_metadata module for core/app/custom properties.

Field	Type	Default	Description
`core_properties`	`Option<CoreProperties>`	`Default::default()`	Core properties from docProps/core.xml (Dublin Core metadata) Contains title, creator, subject, keywords, dates, etc. Shared format across DOCX/PPTX/XLSX documents.
`app_properties`	`Option<DocxAppProperties>`	`Default::default()`	Application properties from docProps/app.xml (Word-specific statistics) Contains word count, page count, paragraph count, editing time, etc. DOCX-specific variant of Office application properties.
`custom_properties`	`Option<HashMap<String, serde_json::Value>>`	`HashMap::new()`	Custom properties from docProps/custom.xml (user-defined properties) Contains key-value pairs defined by users or applications. Values can be strings, numbers, booleans, or dates.

Drawing

A drawing object extracted from <w:drawing>.

Field	Type	Default	Description
`drawing_type`	`DrawingType`	`DrawingType::Inline`	Whether the drawing is inline (in text flow) or anchored (floating).
`extent`	`Option<Extent>`	`Default::default()`	Physical dimensions in EMUs (English Metric Units).
`doc_properties`	`Option<DocProperties>`	`Default::default()`	Document properties such as ID, name, and alt-text description.
`image_ref`	`Option<String>`	`Default::default()`	Relationship ID (`r:embed`) referencing the image part in the DOCX package.
`text_box_content`	`Option<String>`	`Default::default()`	Text extracted from a text box hosted by this drawing (#81): either the DrawingML `wps:txbx/w:txbxContent` path, or the VML `v:textbox/w:txbxContent` fallback path parsed via `parse_vml_pict`.

Element

Semantic element extracted from document.

Represents a logical unit of content with semantic classification, unique identifier, and metadata for tracking origin and position.

Field	Type	Default	Description
`element_id`	`String`	`/* serde(default) */`	Deterministic element identifier. Empty only when deserializing legacy payloads that predate this field’s wire representation.
`element_type`	`ElementType`	—	Semantic type of this element
`text`	`String`	—	Text content of the element
`metadata`	`ElementMetadata`	—	Metadata about the element

ElementMetadata

Metadata for a semantic element.

Field	Type	Default	Description
`page_number`	`Option<u32>`	`None`	Page number (1-indexed)
`filename`	`Option<String>`	`None`	Source filename or document name
`coordinates`	`Option<BoundingBox>`	`None`	Bounding box coordinates if available
`element_index`	`Option<usize>`	`None`	Position index in the element sequence
`additional`	`HashMap<String, String>`	—	Additional custom metadata

EmailAttachment

Email attachment representation.

Contains metadata and optionally the content of an email attachment.

Field	Type	Default	Description
`name`	`Option<String>`	`None`	Attachment name (from Content-Disposition header)
`filename`	`Option<String>`	`None`	Filename of the attachment
`mime_type`	`Option<String>`	`None`	MIME type of the attachment
`size`	`Option<usize>`	`None`	Size in bytes
`is_image`	`bool`	—	Whether this attachment is an image
`data`	`Option<Vec<u8>>`	`None`	Attachment data (if extracted). Uses `bytes::Bytes` for cheap cloning of large buffers.

EmailConfig

Configuration for email extraction.

Field	Type	Default	Description
`msg_fallback_codepage`	`Option<u32>`	`Default::default()`	Windows codepage number to use when an MSG file contains no codepage property. Defaults to `None`, which falls back to windows-1252. If an unrecognized or invalid codepage number is supplied (including 0), the behavior silently falls back to windows-1252 — the same as when the MSG file itself contains an unrecognized codepage. No error or warning is emitted. Users should verify output when supplying unusual values. Common values: - 1250: Central European (Polish, Czech, Hungarian, etc.) - 1251: Cyrillic (Russian, Ukrainian, Bulgarian, etc.) - 1252: Western European (default) - 1253: Greek - 1254: Turkish - 1255: Hebrew - 1256: Arabic - 932: Japanese (Shift-JIS) - 936: Simplified Chinese (GBK)

EmailExtractionResult

Email extraction result.

Complete representation of an extracted email message (.eml or .msg) including headers, body content, and attachments.

Field	Type	Default	Description
`subject`	`Option<String>`	`None`	Email subject line
`from_email`	`Option<String>`	`None`	Sender email address
`to_emails`	`Vec<String>`	—	Primary recipient email addresses
`cc_emails`	`Vec<String>`	—	CC recipient email addresses
`bcc_emails`	`Vec<String>`	—	BCC recipient email addresses
`date`	`Option<String>`	`None`	Email date/timestamp
`message_id`	`Option<String>`	`None`	Message-ID header value
`plain_text`	`Option<String>`	`None`	Plain text version of the email body
`html_content`	`Option<String>`	`None`	HTML version of the email body
`content`	`String`	—	Cleaned/processed text content. Aliased as `cleaned_text` for back-compat.
`attachments`	`Vec<EmailAttachment>`	—	List of email attachments
`metadata`	`HashMap<String, String>`	—	Additional email headers and metadata

EmailExtractor

Email message extractor.

Supports: .eml, .msg

Methods

default()

Signature:

pub fn default() -> EmailExtractor

Example:

let result = EmailExtractor::default();

Returns: EmailExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_sync()

Signature:

pub fn extract_sync(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_sync(b"data", "value", &todo!())?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

EmailMetadata

Email metadata extracted from .eml and .msg files.

Includes sender/recipient information, message ID, and attachment list.

Field	Type	Default	Description
`from_email`	`Option<String>`	`Default::default()`	Sender’s email address
`from_name`	`Option<String>`	`Default::default()`	Sender’s display name
`to_emails`	`Vec<String>`	`vec![]`	Primary recipients
`cc_emails`	`Vec<String>`	`vec![]`	CC recipients
`bcc_emails`	`Vec<String>`	`vec![]`	BCC recipients
`message_id`	`Option<String>`	`Default::default()`	Message-ID header value
`attachments`	`Vec<String>`	`vec![]`	List of attachment filenames

EmbeddedChanges

Changes to embedded archive children between two results.

Field	Type	Default	Description
`added`	`Vec<ArchiveEntry>`	`vec![]`	Children present in `b` but not in `a` (matched by `path`).
`removed`	`Vec<ArchiveEntry>`	`vec![]`	Children present in `a` but not in `b` (matched by `path`).
`changed`	`Vec<EmbeddedDiff>`	`vec![]`	Children present in both but with differing content (matched by `path`). Each entry holds the diff of the nested `ExtractedDocument`.

EmbeddedDiff

Diff for a single embedded archive entry that appears in both results.

Field	Type	Default	Description
`path`	`String`	—	Archive-relative path identifying this entry.
`diff`	`ExtractionDiff`	—	The recursive diff of the entry’s extraction result.

EmbeddedFile

Embedded file descriptor extracted from the PDF name tree.

Field	Type	Default	Description
`name`	`String`	—	The filename as stored in the PDF name tree.
`data`	`Vec<u8>`	—	Raw file bytes from the embedded stream (already decompressed by lopdf).
`compressed_size`	`usize`	—	Compressed byte count of the original stream (before decompression). Used by callers to compute the decompression ratio and detect zip-bomb-style attacks that embed a tiny compressed stream expanding to gigabytes of data.
`mime_type`	`Option<String>`	`None`	MIME type if specified in the filespec, otherwise `None`.

EmbeddingBackend

Trait for in-process embedding backend plugins.

Async to match the convention used by other plugin hooks such as OcrBackend and PostProcessor. Host-language bridges (PyO3, napi-rs, Rustler, extendr, magnus, ext-php-rs, C FFI, etc.) wrap their synchronous host callables in spawn_blocking or the equivalent to satisfy the async signature.

Thread safety

Backends must be Send + Sync + 'static. They are stored in Arc<dyn EmbeddingBackend> and called concurrently from xberg’s chunking pipeline. If the backend’s underlying model isn’t thread-safe, the backend itself must serialize access internally (e.g. via Mutex<Inner>).

Contract

embed(texts) MUST return exactly texts.len() vectors, each of length self.dimensions(). The dispatcher in crate::embeddings::embed_texts validates this before returning to downstream consumers; a non-conforming backend surfaces as a XbergError::Validation, not a panic.
embed may be called from any thread. Its future must be Send (enforced by async_trait when #[async_trait] is used on non-WASM targets).
dimensions() is called exactly once at registration, immediately after initialize() succeeds. The returned value is cached by the registry and used for all subsequent shape validation. Lazy-loading implementations can defer model loading into initialize() and report the real dimension afterwards. Later mutations of the backend’s reported dimension are not observed by xberg — implementations that need to change dimension must unregister and re-register.
shutdown() (inherited from Plugin) may be invoked concurrently with an in-flight embed() call. Implementations must tolerate this — e.g. by letting in-flight calls finish using resources held via the Arc<dyn EmbeddingBackend> reference, and only releasing shared state that isn’t needed by embed.

Runtime

The synchronous embed_texts entry uses tokio::task::block_in_place to await the trait’s async embed, which requires a multi-thread tokio runtime. Callers running inside a current_thread runtime (e.g. #[tokio::test] without flavor = "multi_thread", or tokio::runtime::Builder::new_current_thread()) must use embed_texts_async instead, which awaits directly without block_in_place.

Methods

dimensions()

Embedding vector dimension. Must be > 0 and must match the length of every vector returned by embed.

Signature:

pub fn dimensions(&self) -> usize

Example:

let result = instance.dimensions();

Returns: usize

embed()

Embed a batch of texts, returning one vector per input in order.

Errors:

Implementations should return Plugin for backend-specific failures. The dispatcher layers its own validation (length, per-vector dimension) on top.

Signature:

pub async fn embed(&self, texts: Vec<String>) -> Result<Vec<Vec<f32>>, XbergError>

Example:

let result = instance.embed(vec![]).await?;

Parameters:

Name	Type	Required	Description
`texts`	`Vec<String>`	Yes	The texts

Returns: Vec<Vec<f32>>

Errors: Returns Err(XbergError).

EmbeddingBackendRegistry

Registry for embedding backend plugins.

Unlike OcrBackendRegistry, no default backends are registered — embedding backends are always supplied by the host language at runtime.

Entries cache the backend’s dimensions() at registration time. Downstream code should prefer Self::get_with_dimensions over calling backend.dimensions() on each dispatch so that a backend returning an inconsistent dimension post-registration cannot slip through shape validation mid-session.

Methods

default()

Signature:

pub fn default() -> EmbeddingBackendRegistry

Example:

let result = EmbeddingBackendRegistry::default();

Returns: EmbeddingBackendRegistry

EmbeddingConfig

Embedding configuration for text chunks.

Configures embedding generation using ONNX models via the vendored embedding engine. Requires the embeddings feature to be enabled.

Field	Type	Default	Description
`model`	`EmbeddingModelType`	`EmbeddingModelType::Preset { name: "balanced" }`	The embedding model to use (defaults to “gte-modernbert-base” preset if not specified)
`normalize`	`bool`	`true`	Whether to normalize embedding vectors (recommended for cosine similarity)
`batch_size`	`usize`	`32`	Batch size for embedding generation
`show_download_progress`	`bool`	`false`	Show model download progress. When enabled, transfer progress for the model, tokenizer and config files is reported at `info` level on the `xberg::model_download` target while they download (#279). Covers both local backends (ONNX and static/model2vec). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by `EmbeddingModelType::Llm` and `EmbeddingModelType::Plugin`, which download no model.
`cache_dir`	`Option<PathBuf>`	`None`	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows `HF_HUB_CACHE`, `HUGGINGFACE_HUB_CACHE`, `HF_HOME`, XDG, and platform defaults. Prefer those environment variables when configuring the cache process-wide.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for the embedding ONNX model. When set, controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for inference. Defaults to `None` (auto-select per platform).
`max_embed_duration_secs`	`Option<u64>`	`60`	Maximum wall-clock duration (in seconds) for a single `embed()` call when using `EmbeddingModelType::Plugin`. Applies only to the in-process plugin path — protects against hung host-language backends (e.g. a Python callback deadlocked on the GIL, a model stuck on CUDA OOM retries, etc.). On timeout, the dispatcher returns `Plugin` instead of blocking forever. `None` disables the timeout. The default (60 seconds) is conservative for common in-process inference; increase for large batches on slow hardware.
`max_sequence_length`	`Option<usize>`	`None`	Maximum number of tokens fed to the tokenizer before truncation when embedding a chunk with a local ONNX model (Preset/Custom). A chunk longer than this many tokens has its tail dropped before inference, so only the prefix contributes to the stored vector. `None` falls back to 512 (the historical default). The effective value is always capped at the model’s own `model_max_length`, so raising it past what the model supports has no effect — set it to match a long-context model (e.g. 8192 for Jina/Nomic) so long chunks embed in full. Ignored by the `Llm` and `Plugin` model types, which own their own tokenization.

Methods

default()

Signature:

pub fn default() -> EmbeddingConfig

Example:

let result = EmbeddingConfig::default();

Returns: EmbeddingConfig

EmbeddingEngine

Text embedding model with thread-safe inference.

The embed() method takes &self instead of &mut self, allowing it to be shared across threads via Arc<EmbeddingEngine> without mutex contention.

Entity

A single named entity detected in the extracted text.

Field	Type	Default	Description
`category`	`EntityCategory`	—	Canonical category the entity belongs to (PERSON, ORG, LOCATION, etc.).
`text`	`String`	—	Raw mention text exactly as it appeared in the source.
`start`	`u32`	—	Byte-offset span in `ExtractedDocument::content` where the mention starts.
`end`	`u32`	—	Byte-offset span in `ExtractedDocument::content` where the mention ends (exclusive).
`confidence`	`Option<f32>`	`None`	Backend-reported confidence in `[0.0, 1.0]`. `None` when the backend does not expose confidence scores.

EpubExtractor

EPUB format extractor using permissive-licensed dependencies.

Extracts content and metadata from EPUB files (both EPUB2 and EPUB3) using native Rust parsing without GPL-licensed dependencies.

Methods

default()

Signature:

pub fn default() -> EpubExtractor

Example:

let result = EpubExtractor::default();

Returns: EpubExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

EpubMetadata

EPUB metadata (Dublin Core extensions).

Field	Type	Default	Description
`coverage`	`Option<String>`	`Default::default()`	Dublin Core `coverage` field (geographic or temporal scope).
`dc_format`	`Option<String>`	`Default::default()`	Dublin Core `format` field (media type of the resource).
`relation`	`Option<String>`	`Default::default()`	Dublin Core `relation` field (related resource identifier).
`source`	`Option<String>`	`Default::default()`	Dublin Core `source` field (origin resource identifier).
`dc_type`	`Option<String>`	`Default::default()`	Dublin Core `type` field (nature or genre of the resource).
`cover_image`	`Option<String>`	`Default::default()`	Path or identifier of the cover image within the EPUB container.

ErrorMetadata

Error metadata (for batch operations).

Field	Type	Default	Description
`error_type`	`String`	—	Machine-readable error type identifier (e.g. “UnsupportedFormat”).
`message`	`String`	—	Human-readable error description.

ErrorResponse

Error response.

Field	Type	Default	Description
`error_type`	`String`	—	Error type name
`message`	`String`	—	Error message
`traceback`	`Option<String>`	`Default::default()`	Stack trace (if available)
`status_code`	`u16`	—	HTTP status code

ExcelExtractor

Excel spreadsheet extractor using calamine.

Supports: .xlsx, .xlsm, .xlam, .xltm, .xls, .xla, .xlsb, .ods

Methods

default()

Signature:

pub fn default() -> ExcelExtractor

Example:

let result = ExcelExtractor::default();

Returns: ExcelExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

ExcelMetadata

Excel/spreadsheet format metadata.

Identifies the document as a spreadsheet source via the FormatMetadata::Excel discriminant. Sheet count and sheet names are stored inside this struct.

Field	Type	Default	Description
`sheet_count`	`Option<u32>`	`Default::default()`	Number of sheets in the workbook.
`sheet_names`	`Option<Vec<String>>`	`vec![]`	Names of all sheets in the workbook.

ExcelSheet

Single Excel worksheet.

Represents one sheet from an Excel workbook with its content converted to Markdown format and dimensional statistics.

Field	Type	Default	Description
`name`	`String`	—	Sheet name as it appears in Excel
`markdown`	`String`	—	Sheet content converted to Markdown tables
`row_count`	`usize`	—	Number of rows
`col_count`	`usize`	—	Number of columns
`cell_count`	`usize`	—	Total number of non-empty cells
`table_cells`	`Option<Vec<Vec<String>>>`	`None`	Pre-extracted table cells (2D vector of cell values) Populated during markdown generation to avoid re-parsing markdown. None for empty sheets.

ExcelWorkbook

Excel workbook representation.

Contains all sheets from an Excel file (.xlsx, .xls, etc.) with extracted content and metadata.

Field	Type	Default	Description
`sheets`	`Vec<ExcelSheet>`	—	All sheets in the workbook
`metadata`	`HashMap<String, String>`	—	Workbook-level metadata (author, creation date, etc.)
`revisions`	`Option<Vec<DocumentRevision>>`	`/* serde(default) */`	Collaborative-edit revision headers from `xl/revisions/revisionHeaders.xml`. Populated for legacy shared-workbook `.xlsx` files that contain the `xl/revisions/` directory. Each `<header>` element maps to one `DocumentRevision { kind: FormatChange }` carrying the header’s `guid` (→ `revision_id`), `userName` (→ `author`), and `dateTime` (→ `timestamp`). `anchor` and `delta` are `None`/empty for v1 (per-cell log parsing is a follow-up). `None` when `xl/revisions/revisionHeaders.xml` is absent.

Extent

Size in EMUs (English Metric Units, 1 inch = 914400 EMU).

Field	Type	Default	Description
`cx`	`i64`	—	Width in EMU.
`cy`	`i64`	—	Height in EMU.

ExternalRedactionFinding

Since: v1.3

One finding reported by an external content-inspection engine.

Unknown fields are ignored, so an engine’s raw output can be passed as is. Presidio, AWS Comprehend, Azure Language, and GCP DLP payload fields are accepted as aliases. GCP’s ordered likelihood buckets are normalized to evenly spaced scores from 0.0 through 1.0; LIKELIHOOD_UNSPECIFIED maps to 0.5, matching GCP’s documented POSSIBLE default.

Field	Type	Default	Description
`label`	`String`	—	Engine category, surfaced as `PiiCategory::Custom(label)`.
`text`	`Option<String>`	`Default::default()`	Literal value to redact. When absent, it is read from `content` at `start..end` under the requested `RedactionOffsetEncoding`.
`start`	`Option<u32>`	`Default::default()`	Start offset (inclusive) into `content`.
`end`	`Option<u32>`	`Default::default()`	End offset (exclusive) into `content`.
`score`	`Option<f32>`	`Default::default()`	Engine confidence in `[0.0, 1.0]`; filtered by `RedactionConfig::min_score` when set.

ExtractBatchParams

Request parameters for unified batch extraction.

Field	Type	Default	Description
`inputs`	`Vec<serde_json::Value>`	—	Unified extraction inputs: bytes and/or URIs.
`config`	`Option<serde_json::Value>`	`None`	Extraction configuration (JSON object)
`pdf_password`	`Option<String>`	`None`	Password for encrypted PDFs
`response_format`	`Option<String>`	`None`	Wire format for the response: “json” (default) or “toon”

ExtractInput

Unified extraction input for all public extraction entry points.

Field	Type	Default	Description
`kind`	`ExtractInputKind`	`ExtractInputKind::Uri`	Source kind. `bytes` requires `bytes`; `uri` requires `uri`.
`bytes`	`Option<Vec<u8>>`	`None`	Raw bytes for `kind = "bytes"`.
`uri`	`Option<String>`	`None`	Local path, `file://` URI, or HTTP(S) URL for `kind = "uri"`.
`mime_type`	`Option<String>`	`None`	MIME type hint.
`filename`	`Option<String>`	`None`	Filename hint used for MIME detection and metadata.
`config`	`Option<FileExtractionConfig>`	`None`	Per-input extraction overrides.

Methods

default()

Signature:

pub fn default() -> ExtractInput

Example:

let result = ExtractInput::default();

Returns: ExtractInput

from_bytes()

Build a bytes input with a MIME type and optional filename hint.

Signature:

pub fn from_bytes(bytes: Vec<u8>, mime_type: String, filename: Option<String>) -> ExtractInput

Example:

let result = ExtractInput::from_bytes(b"data", "value", Some("value"));

Parameters:

Name	Type	Required	Description
`bytes`	`Vec<u8>`	Yes	The bytes
`mime_type`	`String`	Yes	The mime type
`filename`	`Option<String>`	No	The filename

Returns: ExtractInput

from_uri()

Build a URI input from a local path, file:// URI, or HTTP(S) URL.

Signature:

pub fn from_uri(uri: String) -> ExtractInput

Example:

let result = ExtractInput::from_uri("value");

Parameters:

Name	Type	Required	Description
`uri`	`String`	Yes	The uri

Returns: ExtractInput

ExtractParams

Request parameters for unified extraction.

Field	Type	Default	Description
`input`	`serde_json::Value`	—	Unified extraction input: bytes or URI.
`config`	`Option<serde_json::Value>`	`None`	Extraction configuration (JSON object)
`pdf_password`	`Option<String>`	`None`	Password for encrypted PDFs
`response_format`	`Option<String>`	`None`	Wire format for the response: “json” (default) or “toon”

ExtractedDocument

Document extracted by the core extraction pipeline.

extract and extract_batch return an ExtractionResult envelope whose results field contains these per-document payloads.

Field	Type	Default	Description
`content`	`String`	—	Plain-text representation of the extracted document content.
`mime_type`	`String`	—	MIME type of the source document (e.g. `"application/pdf"`).
`metadata`	`Metadata`	—	Document-level metadata (author, title, dates, format-specific fields).
`extraction_method`	`Option<ExtractionMethod>`	`Default::default()`	Extraction strategy used to produce the returned text. Populated when the extractor can reliably distinguish native text extraction, OCR-only extraction, or mixed native/OCR output.
`tables`	`Vec<Table>`	`vec![]`	Tables extracted from the document, each with structured cell data.
`counts`	`DocumentCounts`	—	Cheap structural counts (pages, tables, images). Always populated by the extraction pipeline, even when the `pages` / `images` collections are `None`. See `DocumentCounts`.
`detected_languages`	`Option<Vec<String>>`	`vec![]`	ISO 639-1 language codes detected in the document content.
`detected_language_confidences`	`Option<Vec<LanguageConfidence>>`	`vec![]`	Structured per-language detection results: confidence, document share, script, and reliability, alongside the ISO-code-only `detected_languages` (#261). One entry per language in `detected_languages`, in the same order. `None` under the same conditions as `detected_languages`: detection disabled, empty input text, or no language met the configured `min_confidence`.
`chunks`	`Option<Vec<Chunk>>`	`vec![]`	Text chunks when chunking is enabled. When chunking configuration is provided, the content is split into overlapping chunks for efficient processing. Each chunk contains the text, optional embeddings (if enabled), and metadata about its position.
`images`	`Option<Vec<ExtractedImage>>`	`vec![]`	Extracted images from the document. When image extraction is enabled via `ImageExtractionConfig`, this field contains all images found in the document with their raw data and metadata. Each image may optionally contain a nested `ocr_result` if OCR was performed.
`pages`	`Option<Vec<PageContent>>`	`vec![]`	Per-page content when page extraction is enabled. When page extraction is configured, the document is split into per-page content with tables and images mapped to their respective pages.
`elements`	`Option<Vec<Element>>`	`vec![]`	Semantic elements when element-based result format is enabled. When result_format is set to ElementBased, this field contains semantic elements with type classification, unique identifiers, and metadata for Unstructured-compatible element-based processing.
`djot_content`	`Option<DjotContent>`	`Default::default()`	Rich Djot content structure (when extracting Djot documents). When extracting Djot documents with structured extraction enabled, this field contains the full semantic structure including: - Block-level elements with nesting - Inline formatting with attributes - Links, images, footnotes - Math expressions - Complete attribute information The `content` field still contains plain text for backward compatibility. Always `None` for non-Djot documents.
`ocr_elements`	`Option<Vec<OcrElement>>`	`vec![]`	OCR elements with full spatial and confidence metadata. When OCR is performed with element extraction enabled, this field contains the structured representation of detected text including: - Bounding geometry (rectangles or quadrilaterals) - Confidence scores (detection and recognition) - Rotation information - Hierarchical relationships (Tesseract only) This field preserves all metadata that would otherwise be lost when converting to plain text or markdown output formats. Only populated when `OcrElementConfig.include_elements` is true.
`document`	`Option<DocumentStructure>`	`Default::default()`	Structured document tree (when document structure extraction is enabled). When `include_document_structure` is true in `ExtractionConfig`, this field contains the full hierarchical representation of the document including: - Heading-driven section nesting - Table grids with cell-level metadata - Content layer classification (body, header, footer, footnote) - Inline text annotations (formatting, links) - Bounding boxes and page numbers Independent of `result_format` — can be combined with Unified or ElementBased.
`extracted_keywords`	`Option<Vec<Keyword>>`	`vec![]`	Extracted keywords when keyword extraction is enabled. When keyword extraction (RAKE or YAKE) is configured, this field contains the extracted keywords with scores, algorithm info, and position data. Previously stored in `metadata.additional["keywords"]`.
`quality_score`	`Option<f64>`	`Default::default()`	Text cleanliness/readability score from quality analysis. A value between 0.0 and 1.0 describing the quality of the text that was retained. This is not a completeness or recall score: clean text can score highly even when an extractor omitted or rejected other content. Inspect `processing_warnings` separately for known degraded or partial extraction. When the text came from OCR and the result carries enough recognized words to judge, this score is additionally capped by the mean OCR recognition confidence. Text that looks clean but that OCR itself had little confidence in therefore cannot score high. A native, non-OCR extraction is not capped. Previously stored in `metadata.additional["quality_score"]`.
`processing_warnings`	`Vec<ProcessingWarning>`	`vec![]`	Non-fatal warnings collected during processing pipeline stages. Captures errors from optional pipeline features (embedding, chunking, language detection, output formatting) that don’t prevent extraction but may indicate degraded or incomplete results. These warnings are independent of `quality_score`, which assesses only retained text. Previously stored as individual keys in `metadata.additional`.
`annotations`	`Option<Vec<PdfAnnotation>>`	`vec![]`	PDF annotations extracted from the document. When annotation extraction is enabled via `PdfConfig::extract_annotations`, this field contains text notes, highlights, links, stamps, and other annotations found in PDF documents.
`children`	`Option<Vec<ArchiveEntry>>`	`vec![]`	Nested extraction results from archive contents. When extracting archives, each processable file inside produces its own full extraction result. Set to `None` for non-archive formats. Use `max_archive_depth` in config to control recursion depth.
`uris`	`Option<Vec<ExtractedUri>>`	`vec![]`	URIs/links discovered during document extraction. Contains hyperlinks, image references, citations, email addresses, and other URI-like references found in the document. Always extracted when present in the source document.
`revisions`	`Option<Vec<DocumentRevision>>`	`vec![]`	Tracked changes embedded in the source document. Populated by per-format extractors that understand change-tracking metadata (DOCX `w:ins`/`w:del`/`w:rPrChange`, ODT `text:change-*`, …). Every extractor defaults to `None` until its format-specific implementation is added. Extractors that do populate this field follow the “accepted-changes” convention: inserted text is present in `content`, deleted text is absent — the revision list is the separate audit trail.
`structured_output`	`Option<serde_json::Value>`	`Default::default()`	Structured extraction output from LLM-based JSON schema extraction. When `structured_extraction` is configured in `ExtractionConfig`, the extracted document content is sent to a VLM with the provided JSON schema. The response is parsed and stored here as a JSON value matching the schema.
`code_intelligence`	`Option<serde_json::Value>`	`Default::default()`	Code intelligence results from tree-sitter analysis. Populated when extracting source code files with the `tree-sitter` feature. Contains metrics, structural analysis, imports/exports, comments, docstrings, symbols, diagnostics, and optionally chunked code segments. Stored as an opaque JSON value so that all language bindings (Go, Java, C#, …) can deserialize it as a raw JSON object rather than a typed struct. The underlying type is `tree_sitter_language_pack::ProcessResult`.
`llm_usage`	`Option<Vec<LlmUsage>>`	`vec![]`	LLM token usage and cost data for all LLM calls made during this extraction. Contains one entry per LLM call. Multiple entries are produced when VLM OCR, structured extraction, or LLM embeddings run during the same extraction. `None` when no LLM was used.
`entities`	`Option<Vec<Entity>>`	`vec![]`	Named entities detected in `content` by the NER post-processor. `None` when no NER backend is configured. Populated by the `xberg-gliner` ONNX backend or the LLM-driven backend (see `crates/xberg/src/text/ner/`).
`summary`	`Option<DocumentSummary>`	`Default::default()`	Summary of `content` produced by the summarisation post-processor. `None` when summarisation is not configured. Populated by the TextRank extractive backend (deterministic, no external service) or by the liter-llm-driven abstractive backend.
`extraction_confidence`	`Option<ExtractionConfidence>`	`Default::default()`	Confidence score computed by the heuristics pipeline. Populated when the `heuristics` feature is enabled and confidence scoring has been performed. Combines text-coverage, OCR aggregate confidence, and schema-compliance into a single `[0, 1]` value. `None` when confidence scoring is not configured or the feature is absent.
`translation`	`Option<Translation>`	`Default::default()`	Translation of `content` produced by the translation post-processor. `None` when translation is not configured.
`page_classifications`	`Option<Vec<PageClassification>>`	`vec![]`	Per-page classifications produced by the page-classification post-processor. `None` when classification is not configured.
`redaction_report`	`Option<RedactionReport>`	`Default::default()`	Audit report of redactions applied by the redaction post-processor. The redaction processor rewrites `content`, `formatted_content`, every chunk’s text, and the textual fields of `entities` / `summary` / `translation` / `page_classifications` in place. This report describes what was found and how it was replaced. `None` when redaction is not configured.
`formulas`	`Vec<Formula>`	`vec![]`	Mathematical formulas recognized in the document. Populated from every source that produces formulas: layout-guided OCR (with geometry), VLM OCR (text only), and markup extraction (DOCX, PPTX, ODT, EPUB, HTML, JATS, LaTeX, Markdown, and related formats, without geometry). Empty when the document contains no formulas.
`form_fields`	`Vec<PdfFormField>`	`vec![]`	Form fields extracted from a PDF’s AcroForm or XFA structure. Populated by the PDF extractor when `PdfConfig::extract_form_fields` is enabled (default) and the document is a fillable form. Empty otherwise.
`ocr_page_failures`	`Option<Vec<OcrPageFailure>>`	`vec![]`	Pages whose OCR failed while extraction continued, in the order the failures were reported. A record with `recovered` set means the document still has content for that page from native text or from embedded images. Xberg-produced results use `None` when no page failed.
`formatted_content`	`Option<String>`	`Default::default()`	Pre-rendered content in the requested output format. Pipeline-internal scratch space, not a result field. `derive_extraction_result` renders it before tree derivation consumes the element data, post-processors may rewrite it alongside `content`, and `apply_output_format` then moves it into `content` as the last pipeline step. Every document returned by `extract_bytes` / `extract_file` — and every nested archive or email child — therefore carries `None` here, with the rendering in `content`. It stays `pub` because it is part of the Rust plugin contract: a `DocumentExtractor` may return it pre-rendered, and a post-processor that rewrites `content` must rewrite this alongside it or the rendering is discarded as stale (see `core::pipeline::discard_diverged_formatted_content`). It is hidden from the language bindings, which only ever observe the post-pipeline document.

ExtractedImage

Extracted image from a document.

Contains raw image data, metadata, and optional nested OCR results. Raw bytes allow cross-language compatibility - users can convert to PIL.Image (Python), Sharp (Node.js), or other formats as needed.

Field	Type	Default	Description
`data`	`Vec<u8>`	—	Raw image data (PNG, JPEG, WebP, etc. bytes). Uses `bytes::Bytes` for cheap cloning of large buffers.
`format`	`String`	—	Image format (e.g., “jpeg”, “png”, “webp”) Uses Cow<’static, str> to avoid allocation for static literals.
`image_index`	`u32`	—	Zero-indexed position of this image in the document/page
`page_number`	`Option<u32>`	`Default::default()`	Page/slide number where image was found (1-indexed)
`width`	`Option<u32>`	`Default::default()`	Image width in pixels
`height`	`Option<u32>`	`Default::default()`	Image height in pixels
`colorspace`	`Option<String>`	`Default::default()`	Colorspace information (e.g., “RGB”, “CMYK”, “Gray”)
`bits_per_component`	`Option<u32>`	`Default::default()`	Bits per color component (e.g., 8, 16)
`is_mask`	`bool`	—	Whether this image is a mask image
`description`	`Option<String>`	`Default::default()`	Optional description of the image
`ocr_result`	`Option<ExtractedDocument>`	`Default::default()`	Nested OCR extraction result (if image was OCRed) When OCR is performed on this image, the result is embedded here rather than in a separate collection, making the relationship explicit.
`bounding_box`	`Option<BoundingBox>`	`Default::default()`	Bounding box of the image on the page (PDF coordinates: x0=left, y0=bottom, x1=right, y1=top). Only populated for PDF-extracted images when position data is available from the PDF extractor.
`source_path`	`Option<String>`	`Default::default()`	Original source path of the image within the document archive (e.g., “media/image1.png” in DOCX). Used for rendering image references when the binary data is not extracted.
`image_kind`	`Option<ImageKind>`	`Default::default()`	Heuristic classification of what this image likely depicts. `None` if classification was disabled or inconclusive.
`kind_confidence`	`Option<f32>`	`Default::default()`	Confidence score for `image_kind`, in the range 0.0 to 1.0.
`cluster_id`	`Option<u32>`	`Default::default()`	Identifier shared across images that form a single logical figure (e.g. all raster tiles of one technical drawing). `None` for singletons.
`caption`	`Option<String>`	`Default::default()`	VLM-generated caption describing the image, when captioning is configured. Populated by the captioning post-processor (`crates/xberg/src/plugins/processor/builtin/captioning.rs`), which routes each image through `crate::llm::region_extractor::extract_region_with_vlm` in caption mode. `None` when captioning is disabled or the VLM declined to caption.
`qr_codes`	`Option<Vec<QrCode>>`	`vec![]`	QR codes decoded from this image, when QR detection is enabled. Populated by the QR post-processor (`crates/xberg/src/extractors/qr.rs`) via the pure-Rust `rqrr` decoder. `None` when QR detection is disabled; an empty `Some(vec![])` when detection ran but found nothing.
`data_base64`	`Option<String>`	`Default::default()`	Base64-encoded copy of `data`; populated when `ImageExtractionConfig::include_data_base64` is `true`. Omitted from JSON by default; use instead of `data` in JSON-only clients.

ExtractedInlineImage

Extracted inline image with metadata.

Field	Type	Default	Description
`data`	`Vec<u8>`	—	Raw image bytes (shared via `Bytes` for cheap cloning).
`format`	`String`	—	Image format string (e.g. `"png"`, `"jpeg"`, `"gif"`).
`filename`	`Option<String>`	`None`	Optional original filename from `src` or `data-filename` attribute.
`description`	`Option<String>`	`None`	Alt-text or title used as the image description.
`dimensions`	`Option<(u32, u32)>`	`None`	Image dimensions as `(width, height)` in pixels, if known.
`attributes`	`Vec<(String, String)>`	—	All HTML attributes from the `<img>` element.

ExtractedUri

A URI extracted from a document.

Represents any link, reference, or resource pointer found during extraction. The kind field classifies the URI semantically, while label carries optional human-readable display text.

Field	Type	Default	Description
`url`	`String`	—	The URL or path string.
`label`	`Option<String>`	`None`	Optional display text / label for the link.
`page`	`Option<u32>`	`None`	Optional page number where the URI was found (1-indexed).
`kind`	`UriKind`	—	Semantic classification of the URI.

ExtractionConfidence

Combined confidence on [0, 1].

When OCR did not run, the ocr_aggregate weight folds into text_coverage so the weighted sum still totals 1.0.

Field	Type	Default	Description
`text_coverage`	`f32`	—	Fraction of pages with a usable text layer.
`ocr_aggregate`	`Option<f32>`	`None`	OCR recognition confidence, word-count-weighted across every recognized word, when OCR ran; `None` when it did not.
`schema_compliance`	`SchemaCompliance`	—	Whether the merged output validates against the preset schema.
`combined`	`f32`	—	Weighted blend in `[0, 1]`. The value compared against the fallback threshold.

ExtractionConfig

Main extraction configuration.

This struct contains all configuration options for the extraction process. It can be loaded from TOML, YAML, or JSON files, or created programmatically.

Field	Type	Default	Description
`mime_detection_policy`	`MimeDetectionPolicy`	`MimeDetectionPolicy::PreferContent`	Controls whether MIME inference prefers content, a supported extension, or content alone.
`use_cache`	`bool`	`true`	Enable caching of extraction results
`enable_quality_processing`	`bool`	`true`	Enable quality post-processing
`ocr`	`Option<OcrConfig>`	`None`	OCR configuration. `None` does not run OCR for documents that already have usable text. Under `OcrStrategy::Auto`, a PDF with no text layer at all (a scan) is still routed to OCR with default settings so it is not returned empty (#1338). Set `Self::disable_ocr` to hard-disable OCR regardless of the detected content.
`force_ocr`	`bool`	`false`	Force OCR even for searchable PDFs
`ocr_strategy`	`OcrStrategy`	`OcrStrategy::Auto`	Which pages get OCR’d when neither `force_ocr` nor `force_ocr_pages` applies. Defaults to `OcrStrategy::Auto`, which OCRs only pages whose native text fails a quality check. Only applies to PDF documents. Cannot be `OcrStrategy::ScannedPages` while `disable_ocr` is `true`.
`force_ocr_pages`	`Option<Vec<u32>>`	`None`	Force OCR on specific pages only (1-indexed page numbers, must be >= 1). When set, only the listed pages are OCR’d regardless of text layer quality. Unlisted pages use native text extraction. Ignored when `force_ocr` is `true`. Only applies to PDF documents. Duplicates are automatically deduplicated. An `ocr` config is recommended for backend/language selection; defaults are used if absent.
`disable_ocr`	`bool`	`false`	Disable OCR entirely, even for images. When `true`, OCR is skipped for all document types. Images return metadata only (dimensions, format, EXIF) without text extraction. PDFs use only native text extraction without OCR fallback. Cannot be `true` simultaneously with `force_ocr`.
`ocr_near_empty_fallback`	`Option<bool>`	`None`	Whether a PDF whose native text layer is empty or near-empty falls back to OCR under `OcrStrategy::Auto` (GH#1752). `None` (the default) derives the answer from whether `Self::ocr` is set, which is exactly the behaviour shipped before this field existed: with an `ocr` block the fallback always runs, and without one it runs only when the page’s native text is completely empty (#1338). That carve-out means a scanned page carrying a visible page label, Bates number or scanner stamp keeps only that label. `Some(true)` applies the near-empty fallback without requiring an `ocr` block: the text-quality gate decides, using `OcrConfig::default`’s thresholds when no block is present, and an automatic OCR backend must be registered. `Some(false)` suppresses the fallback even when an `ocr` block is present.
`ocr_scanned_page_quality_gate`	`Option<bool>`	`None`	Whether `OcrStrategy::ScannedPages` folds the per-page text-quality gate into its page selection, on top of the pages scan detection flagged (GH#1752). `None` (the default) derives the answer from whether `Self::ocr` is set, which is exactly the behaviour shipped before this field existed. Without an `ocr` block `ScannedPages` therefore degrades to detected-scans-only. `Some(true)` runs the gate without requiring an `ocr` block, using `OcrConfig::default`’s thresholds; an automatic OCR backend must be registered. `Some(false)` selects detected scans only even when an `ocr` block is present, which is the setting that avoids paying for recognition across a whole mixed document.
`ocr_embedded_images`	`Option<bool>`	`None`	Whether images embedded in a container document (DOCX, PPTX, ODT, HTML, …) are sent to OCR (GH#1752). `None` (the default) derives the answer from whether `Self::ocr` is set, which is exactly the behaviour shipped before this field existed. `Some(true)` recognises picture text without requiring an `ocr` block; `Some(false)` suppresses it even when a block is present. Orthogonal to `ImageExtractionConfig::run_ocr_on_images`, which still has to be `true` (its own default) for embedded-image OCR to run. See `Self::runs_ocr_on_embedded_images`.
`chunking`	`Option<ChunkingConfig>`	`None`	Text chunking configuration (None = chunking disabled)
`content_filter`	`Option<ContentFilterConfig>`	`None`	Content filtering configuration (None = use extractor defaults). Controls whether document “furniture” (headers, footers, watermarks, repeating text) is included in or stripped from extraction results. See `ContentFilterConfig` for per-field documentation.
`images`	`Option<ImageExtractionConfig>`	`None`	Image extraction configuration (None = no image extraction)
`pdf_options`	`Option<PdfConfig>`	`None`	PDF-specific options (None = use defaults)
`token_reduction`	`Option<TokenReductionOptions>`	`None`	Token reduction configuration (None = no token reduction)
`language_detection`	`Option<LanguageDetectionConfig>`	`None`	Language detection configuration (None = no language detection)
`pages`	`Option<PageConfig>`	`None`	Page extraction configuration (None = no page tracking)
`keywords`	`Option<KeywordConfig>`	`None`	Keyword extraction configuration (None = no keyword extraction)
`postprocessor`	`Option<PostProcessorConfig>`	`None`	Post-processor configuration (None = use defaults)
`html_options`	`Option<ConversionOptions>`	`None`	HTML to Markdown conversion options (None = use defaults) Configure how HTML documents are converted to Markdown, including heading styles, list formatting, code block styles, and preprocessing options.
`html_output`	`Option<HtmlOutputConfig>`	`None`	Styled HTML output configuration. When set alongside `output_format = OutputFormat::Html`, the extraction pipeline uses `StyledHtmlRenderer` which emits stable `kb-*` CSS class hooks on every structural element and optionally embeds theme CSS or user-supplied CSS in a `<style>` block. When `None`, the existing plain comrak-based HTML renderer is used.
`extraction_timeout_secs`	`Option<u64>`	`600`	Default per-file timeout in seconds for batch extraction. When set, each file in a batch will be canceled after this duration unless overridden by `FileExtractionConfig::timeout_secs`. Defaults to `Some(600)` (10 minutes) to prevent pathological files (e.g. deeply nested archives, documents with millions of cells) from running indefinitely and exhausting caller resources, while still giving slow paths (VLM-based OCR, large scanned documents) enough headroom to finish. Set to `None` to disable the timeout for trusted input or long-running workloads.
`max_concurrent_extractions`	`Option<usize>`	`None`	Maximum concurrent document extractions in batch operations. This is a ceiling within the configured total thread budget, not an independent pool size. When unset, the scheduler derives document and per-document concurrency from `ConcurrencyConfig::max_threads`.
`result_format`	`ResultFormat`	`ResultFormat::Unified`	Result structure format Controls whether results are returned in unified format (default) with all content in the `content` field, or element-based format with semantic elements (for Unstructured-compatible output).
`security_limits`	`Option<SecurityLimits>`	`None`	Security limits for archive extraction. Controls maximum archive size, compression ratio, file count, and other security thresholds to prevent decompression bomb attacks. Also caps nesting depth, iteration count, entity / token length, total content size, decoded image allocation, and table cell count for every extraction path that ingests user-controlled bytes. When `None`, default limits are used.
`max_embedded_file_bytes`	`Option<u64>`	`52428800`	Maximum uncompressed size in bytes for a single embedded file before recursive extraction is attempted (default: 50 MiB). Applies to embedded objects inside OOXML containers (DOCX, PPTX) and to email attachments processed via recursive extraction. Files that exceed this limit are skipped with a `ProcessingWarning` rather than passed to the extraction pipeline, preventing a single oversized embedded object from consuming unbounded memory or time. Set to `None` to disable the per-embedded-file cap (falls back to `security_limits.max_archive_size` as the only guard).
`output_format`	`OutputFormat`	`OutputFormat::Plain`	Content text format (default: Plain). Controls the format of the extracted content: - `Plain`: Raw extracted text (default) - `Markdown`: Markdown formatted output - `Djot`: Djot markup format (requires djot feature) - `Html`: HTML formatted output When set to a structured format, extraction results will include formatted output. The `formatted_content` field may be populated when format conversion is applied.
`escape_markdown`	`bool`	`true`	Escape Markdown special characters in rendered prose (default: `true`). When `output_format` is `Markdown` or `Djot`, the renderer backslash-escapes CommonMark-significant leading characters (e.g. `-`, `#`) so that literal text such as `#06-18` or `- clause` round-trips safely through a CommonMark parser instead of being reinterpreted as a heading or list marker. Table cell text is never escaped, so escaped prose can look inconsistent with table cells containing the same characters. Set this to `false` to disable prose escaping and make `content`, `pages[].content`, and `chunks[].content` read identically to table cell text — useful for LLM prompts or search indexing where CommonMark round-tripping does not matter. Defaults to `true` to preserve existing behavior.
`table_anchors`	`bool`	`false`	Emit an opt-in anchor marker before each table’s rendered Markdown block (default: `false`). When `output_format` is `Markdown` (or `Djot`) and this is `true`, the renderer inserts a `[TABLE:{table_id}]` marker immediately before each table’s Markdown in `content`, `pages[].content`, and `chunks[].content`, where `table_id` matches the corresponding entry’s `table_id`. This lets a consumer reconcile a rendered Markdown table block with its structured `tables[]` entry. Defaults to `false` so existing output is byte-identical unless explicitly enabled.
`jupyter_cell_rendering`	`JupyterCellRendering`	`JupyterCellRendering::Both`	Controls how Jupyter notebook (`.ipynb`) code cells are rendered. - `Both` (default): code source plus the notebook’s saved outputs - `Source`: only the code source (fenced code blocks) - `Outputs`: only the saved outputs Cells are never executed; `Outputs`/`Both` surface only outputs already stored in the notebook.
`apply_notebook_cell_tags`	`bool`	`true`	Apply Jupyter Book/MyST cell visibility tags while rendering notebooks. When enabled, `remove-cell`/`hide-cell`, `remove-input`/`hide-input`, and `remove-output`/`hide-output` suppress the corresponding saved source or output. Cells are never executed, and their metadata remains available even when their rendered content is suppressed. Defaults to `true`. Set this to `false` to preserve all saved notebook content regardless of cell tags.
`layout`	`Option<LayoutDetectionConfig>`	`None`	Layout detection configuration (None = layout detection disabled). When set, PDF pages and images are analyzed for document structure (headings, code, formulas, tables, figures, etc.) using RT-DETR models via ONNX Runtime. For PDFs, layout hints override paragraph classification in the markdown pipeline. For images, per-region OCR is performed with markdown formatting based on detected layout classes. Requires the `layout-detection` feature to run inference; the field is present whenever the `layout-types` feature is active (which includes `layout-detection` as well as the no-ORT target groups).
`transcription`	`Option<TranscriptionConfig>`	`None`	Transcription (speech-to-text) configuration for audio/video files. When set and `enabled`, files with audio/video MIME types (mp3, mp4, m4a, wav, webm, etc.) are routed to the Whisper-based transcription pipeline. The actual heavy dependencies are only active under the `transcription` feature; the field is visible under `transcription-types` (including on WASM and Android targets that use the no-ORT preset). Default: `None` (transcription disabled). This is an additive, non-breaking change.
`use_layout_for_markdown`	`bool`	`false`	Run layout detection on the non-OCR PDF markdown path. When `true` and `layout` is `Some(_)`, layout regions inform reading order, region grouping, and table detection while native font/tag semantics remain authoritative for headings, lists, code, and formulas. OCR layout classification is unchanged. This improves structural output at the cost of inference latency (~150-300ms/page CPU, ~20-50ms/page GPU). Default: `false`. Requires the `layout-detection` feature.
`include_document_structure`	`bool`	`false`	Enable structured document tree output. When true, populates the `document` field on `ExtractedDocument` with a hierarchical `DocumentStructure` containing heading-driven section nesting, table grids, content layer classification, and inline annotations. Independent of `result_format` — can be combined with Unified or ElementBased.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration configuration for ONNX Runtime models. Controls execution provider selection for layout detection and embedding models. When `None`, uses platform defaults (CoreML on macOS, CUDA on Linux, CPU on Windows).
`cache_namespace`	`Option<String>`	`None`	Cache namespace for tenant isolation. When set, cache entries are stored under `{cache_dir}/{namespace}/`. Must be alphanumeric, hyphens, or underscores only (max 64 chars). Different namespaces have isolated cache spaces on the same filesystem.
`cache_ttl_secs`	`Option<u64>`	`None`	Per-request cache TTL in seconds. Overrides the global `max_age_days` for this specific extraction. When `0`, caching is completely skipped (no read or write). When `None`, the global TTL applies.
`email`	`Option<EmailConfig>`	`None`	Email extraction configuration (None = use defaults). Currently supports configuring the fallback codepage for MSG files that do not specify one. See `EmailConfig` for details.
`csv`	`Option<CsvConfig>`	`None`	CSV/TSV extraction configuration (None = use defaults). Lets callers set an explicit delimiter and declare comment-line prefixes to skip, instead of relying solely on delimiter auto-detection. See `CsvConfig` for details.
`geojson`	`Option<GeoJsonExtractionConfig>`	`None`	GeoJSON extraction configuration (None = bounded summary). By default, GeoJSON coordinates are replaced by aggregate counts and bounds so large geometry arrays do not become unbounded rendered output. Set `include_full_coordinates` explicitly to retain the legacy full-coordinate output.
`concurrency`	`Option<ConcurrencyConfig>`	`None`	Concurrency limits for constrained environments (None = use defaults). Controls Rayon thread pool size, ONNX Runtime intra-op threads, and the combined document/inner-task budget for batch extraction. See `ConcurrencyConfig` for details.
`url`	`UrlExtractionConfig`	—	URL ingestion and crawl configuration.
`max_archive_depth`	`usize`	`3`	Maximum recursion depth for archive extraction (default: 3). Set to 0 to disable recursive extraction (legacy behavior).
`tree_sitter`	`Option<TreeSitterConfig>`	`None`	Tree-sitter language pack configuration (None = tree-sitter disabled). When set, enables code file extraction using tree-sitter parsers. Controls grammar download behavior and code analysis options.
`structured_extraction`	`Option<StructuredExtractionConfig>`	`None`	Structured extraction via LLM (None = disabled). When set, the extracted document content is sent to an LLM with the provided JSON schema. The structured response is stored in `ExtractedDocument::structured_output`.
`ner`	`Option<NerConfig>`	`None`	Named-entity recognition configuration. When set, the NER post-processor runs at the Middle stage and populates `ExtractedDocument::entities`.
`redaction`	`Option<RedactionConfig>`	`None`	Redaction / anonymisation configuration. When set, the redaction post-processor runs at the Late stage and rewrites every textual field in `ExtractedDocument`, emitting an audit trail in `ExtractedDocument::redaction_report`.
`summarization`	`Option<SummarizationConfig>`	`None`	Summarisation configuration. When set, the summarisation post-processor runs at the Middle stage and populates `ExtractedDocument::summary`.
`translation`	`Option<TranslationConfig>`	`None`	Translation configuration. When set, the translation post-processor runs at the Middle stage and populates `ExtractedDocument::translation`.
`page_classification`	`Option<PageClassificationConfig>`	`None`	Per-page classification configuration. When set, the classification post-processor runs at the Middle stage and populates `ExtractedDocument::page_classifications`.
`chunk_classification`	`Option<ChunkClassificationConfig>`	`None`	Per-chunk multi-label classification configuration. When set, the chunk-classification post-processor runs at the Middle stage (after chunking) and populates `ChunkMetadata::classifications` on every chunk.
`captioning`	`Option<CaptioningConfig>`	`None`	VLM captioning configuration for extracted images. When set, the captioning post-processor runs at the Middle stage and writes a caption into each `ExtractedImage::caption`.
`qr_codes`	`Option<bool>`	`None`	Enable QR-code detection in extracted images. When `true`, the QR post-processor runs at the Middle stage and populates `ExtractedImage::qr_codes`.
`cancel_token`	`Option<CancellationToken>`	`None`	Cooperative cancellation handle (`None` = no caller-initiated cancellation). Rust callers may supply a `CancellationToken`, retain a clone, and call `cancel` to request that extraction stop at its next cancellation checkpoint. Cancellation is cooperative rather than immediate, so latency depends on the extractor and operation currently in progress. Xberg also installs and uses cancellation tokens internally for extraction timeouts and REST async jobs. This field remains excluded from serialization and Alef-generated language bindings.
`source_name`	`Option<String>`	`None`	Transient source-filename hint for extension-based language detection. Set internally during extraction (from `ExtractInput::filename` or a downloaded document’s filename) so extractors such as the tree-sitter code extractor can fall back to extension-based detection when content-based detection (e.g. shebang) is inconclusive. Excluded from serialization and bindings — it is not a user-facing configuration value. `pub` (not `pub(crate)`) so binding crates can construct `ExtractionConfig` via struct-update syntax (`..the default constructor`); a single private field would make that construction illegal across crates (E0451), matching the existing `cancel_token` precedent.

Methods

default_archive_depth()

Default Self::max_archive_depth: 3 levels of nested archives.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_archive_depth() -> usize

Example:

let result = ExtractionConfig::default_archive_depth();

Returns: usize

default_max_embedded_file_bytes()

Default per-embedded-file cap: 50 MiB.

A single embedded object larger than this can consume significant memory when the recursive extractor materialises it. 50 MiB is generous for real-world embedded documents while still bounding worst-case allocation.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_max_embedded_file_bytes() -> Option<u64>

Example:

let result = ExtractionConfig::default_max_embedded_file_bytes();

Returns: Option<u64>

default_extraction_timeout()

Default extraction timeout: 600 seconds (10 minutes).

Pathological files (deeply nested archives, sheets with millions of cells, adversarial PDFs) can otherwise run indefinitely and exhaust caller resources. 600 s bounds the worst-case cost of a single untrusted input while giving legitimate but slow paths — VLM-based OCR, large scanned documents — enough headroom to finish instead of being cut off at the previous 60 s default.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_extraction_timeout() -> Option<u64>

Example:

let result = ExtractionConfig::default_extraction_timeout();

Returns: Option<u64>

default()

Signature:

pub fn default() -> ExtractionConfig

Example:

let result = ExtractionConfig::default();

Returns: ExtractionConfig

validate()

Validate the configuration, returning an error if any settings are invalid.

Checks:

ocr: backend name, VLM backend/model requirements, language codes, and the vlm_fallback quality threshold.
chunking: max_characters is non-zero and overlap is smaller than it. topic_threshold, when set, is a finite [0.0, 1.0] value.
token_reduction: mode is one of the recognized reduction levels.
images: target_dpi, min_dpi, and max_dpi are all positive and within the supported range.
language_detection: min_confidence is a [0.0, 1.0] value.
csv: delimiter, when set, is exactly one ASCII character.
keywords: the n-gram range contains positive, ordered bounds and min_score is a finite [0.0, 1.0] value.
layout: confidence_threshold, when set, is a finite [0.0, 1.0] value.
redaction: custom terms, patterns, and external findings are valid.
pdf_options: hierarchy cluster counts and margin fractions are in their supported ranges.
every nested LLM config: sampling ranges and target-specific authentication support.

Called automatically when a config is loaded from a file, built from a JSON override, or passed to the public extract and extract_batch entry points. Call this method explicitly before passing a typed config to lower-level processing APIs.

Errors:

Returns XbergError::Validation describing the first invalid setting found.

Signature:

pub fn validate(&self) -> Result<(), XbergError>

Example:

instance.validate()?;

Returns: No return value.

Errors: Returns Err(XbergError).

needs_image_data()

Check if image processing is needed by examining OCR and image extraction settings.

Returns true if either OCR is enabled or image extraction is configured, indicating that image decompression and processing should occur. Returns false if both are disabled, allowing optimization to skip unnecessary image decompression for text-only extraction workflows.

Optimization Impact

For text-only extractions (no OCR, no image extraction), skipping image decompression can improve CPU utilization by 5-10% by avoiding wasteful image I/O and processing when results won’t be used. Returns true when a CONTAINER document (DOCX, PPT, PPTX, HTML, …) needs to read an embedded image’s binary data out of its archive or source.

True when config.images.extract_images is set, captioning is configured, QR-code detection is enabled, or embedded images get OCR’d. All four consume the bytes: captioning and QR-code detection regardless of whether the caller also requested image extraction, and embedded-image OCR because the OCR path reads the same bytes the container would otherwise skip (GH#1662): before the OCR disjunct existed, a container asked this question, was told no, and attached an image with an empty buffer, which OCR then ran on and reported Could not determine image format.

This governs whether a CONTAINER reads bytes it does not otherwise have. A standalone image extractor already holds the whole file’s bytes regardless of this answer, so it must not use this method to decide whether to put its OWN bytes in the result’s images array; that is wants_own_bytes_in_result.

Signature:

pub fn needs_image_data(&self) -> bool

Example:

let result = instance.needs_image_data();

Returns: bool

runs_ocr_on_embedded_images()

Whether embedded images get OCR’d.

This is THE condition – core/pipeline/mod.rs and extraction::image_ocr::process_images_with_ocr both call this method rather than re-deriving it, because when two copies of it drifted apart a container extractor asked needs_image_data and was told no, so it attached an image with an empty buffer, and the OCR path then ran on those zero bytes and reported Could not determine image format (GH#1662). A re-derived copy anywhere reopens that defect; embedded_image_ocr_gate_has_no_second_copy in core/pipeline/tests.rs fails if one appears.

Self::ocr_embedded_images is the caller’s explicit answer to the OCR half; None derives it from whether an ocr block is present, which is what this condition was before GH#1752 gave the behaviour a setting of its own.

disable_ocr still wins regardless: it is documented as skipping OCR “for all document types”, and ocr_embedded_images (like the plain presence of an ocr block before it) is an AUTOMATIC trigger, not an explicit request like force_ocr – see Self::effective_disable_ocr’s callers elsewhere (needs_image_processing, extractors/image.rs, engine/extract_impl.rs) for the same precedent.

Signature:

pub fn runs_ocr_on_embedded_images(&self) -> bool

Example:

let result = instance.runs_ocr_on_embedded_images();

Returns: bool

wants_own_bytes_in_result()

Returns true when a standalone image extraction (the whole input document IS the image, not a container’s embedded picture) should attach its own already-read bytes to the result’s images array.

Deliberately narrower than needs_image_data: a standalone image’s content is read in full before this question is ever asked, whether or not OCR runs, so OCR being configured is not by itself a reason to echo those bytes back in the public output. Using needs_image_data here (as needs_image_data’s own OCR disjunct briefly did) would silently start returning every OCR’d standalone image’s raw bytes in images, a public output contract change with nothing asking for it. Same three conditions as before that disjunct existed: explicit image extraction, captioning, or QR-code detection.

Signature:

pub fn wants_own_bytes_in_result(&self) -> bool

Example:

let result = instance.wants_own_bytes_in_result();

Returns: bool

needs_image_processing()

Returns true when any image processing is needed during extraction.

Optimization Impact

For text-only extractions (no OCR, no image extraction, no captioning), skipping image decompression can improve CPU utilization by 5-10% by avoiding wasteful image I/O and processing when results won’t be used.

Signature:

pub fn needs_image_processing(&self) -> bool

Example:

let result = instance.needs_image_processing();

Returns: bool

ExtractionDiff

The complete diff between two ExtractedDocument values.

Field	Type	Default	Description
`content_diff`	`Vec<DiffHunk>`	`vec![]`	Unified-diff hunks for the `content` field. Empty when the content is identical.
`tables_added`	`Vec<Table>`	`vec![]`	Tables present in `b` but not in `a` (by index position, excess right-side tables).
`tables_removed`	`Vec<Table>`	`vec![]`	Tables present in `a` but not in `b` (by index position, excess left-side tables).
`tables_changed`	`Vec<TableDiff>`	`vec![]`	Cell-level changes for table pairs that share the same index and dimensions.
`metadata_changed`	`serde_json::Value`	—	Metadata difference, encoded as a JSON object with three top-level keys: `added` (keys present in `b` but not `a`), `removed` (keys present in `a` but not `b`), and `changed` (keys whose values differ — each entry is `{ "from": <value-in-a>, "to": <value-in-b> }`). This is NOT RFC 6902 JSON Patch — we deliberately chose a flatter shape to avoid pulling in a json-patch crate. If you need RFC 6902 semantics (with JSON Pointer paths) feed `a.metadata` and `b.metadata` to your preferred json-patch impl directly.
`embedded_changes`	`EmbeddedChanges`	—	Changes to embedded archive children.

ExtractionErrorItem

Non-fatal per-input extraction error captured by ExtractionResult.

Field	Type	Default	Description
`index`	`usize`	—	Input index in the original request.
`code`	`u32`	—	Stable numeric error code.
`error_type`	`String`	—	Stable snake_case error kind.
`source`	`String`	—	Best-effort source identifier.
`message`	`String`	—	Error message.

ExtractionMetrics

Collection of all xberg metric instruments.

Field	Type	Default	Description
`extraction_total`	`Counter`	—	Total extractions (attributes: mime_type, extractor, status).
`cache_hits`	`Counter`	—	Cache hits.
`cache_misses`	`Counter`	—	Cache misses.
`batch_total`	`Counter`	—	Total batch requests (attributes: status).
`extraction_duration_ms`	`Histogram`	—	Extraction wall-clock duration in milliseconds (attributes: mime_type, extractor).
`extraction_input_bytes`	`Histogram`	—	Input document size in bytes (attributes: mime_type).
`extraction_output_bytes`	`Histogram`	—	Output content size in bytes (attributes: mime_type).
`pipeline_duration_ms`	`Histogram`	—	Pipeline stage duration in milliseconds (attributes: stage).
`ocr_duration_ms`	`Histogram`	—	OCR duration in milliseconds (attributes: backend, language).
`batch_duration_ms`	`Histogram`	—	Batch total duration in milliseconds.
`concurrent_extractions`	`UpDownCounter`	—	Currently in-flight extractions.

ExtractionRequest

A request to extract content from a single document.

Field	Type	Default	Description
`source`	`ExtractionSource`	—	Where to read the document from.
`config`	`ExtractionConfig`	—	Base extraction configuration.
`file_overrides`	`Option<FileExtractionConfig>`	`None`	Optional per-file overrides (merged on top of `config`).

ExtractionResult

Unified extraction result envelope.

Field	Type	Default	Description
`results`	`Vec<ExtractedDocument>`	`vec![]`	Extracted documents in discovery order.
`errors`	`Vec<ExtractionErrorItem>`	`vec![]`	Non-fatal per-input errors.
`summary`	`ExtractionSummary`	—	Aggregate counts for the operation.
`crawl_final_urls`	`Vec<String>`	`vec![]`	Final URLs reached after redirects during URL ingestion.
`crawl_redirect_count`	`usize`	—	Total redirects followed while fetching or crawling URLs.
`crawl_unique_normalized_urls`	`Vec<String>`	`vec![]`	Unique normalized URLs discovered by crawls.

Methods

single()

Build an output containing one successful result.

Signature:

pub fn single(result: ExtractedDocument) -> ExtractionResult

Example:

let result = ExtractionResult::single(todo!());

Parameters:

Name	Type	Required	Description
`result`	`ExtractedDocument`	Yes	The extracted document

Returns: ExtractionResult

ExtractionResult2

OCR extraction result returned by the internal OCR processor.

Field	Type	Default	Description
`content`	`String`	—	Extracted text content (plain text, Markdown, or hOCR depending on `output_format`).
`mime_type`	`String`	—	MIME type of the source image.
`metadata`	`HashMap<String, serde_json::Value>`	—	Additional metadata key-value pairs (e.g. page count, engine version).
`tables`	`Vec<Table>`	—	Tables reconstructed from the OCR output.

ExtractionService

A tower::Service that dispatches extraction requests to the xberg core library.

This service is cheap to clone and can be shared across handlers. Concurrency and timeouts are managed by composing Tower layers on top (see ExtractionServiceBuilder).

Methods

default()

Signature:

pub fn default() -> ExtractionService

Example:

let result = ExtractionService::default();

Returns: ExtractionService

poll_ready()

Signature:

pub fn poll_ready(&mut self, cx: &mut Context) -> String

Example:

let result = instance.poll_ready(&mut todo!());

Parameters:

Name	Type	Required	Description
`cx`	`&mut Context`	Yes	The cx

Returns: String

call()

Signature:

pub fn call(&mut self, req: ExtractionRequest) -> String

Example:

let result = instance.call(todo!());

Parameters:

Name	Type	Required	Description
`req`	`ExtractionRequest`	Yes	The extraction request

Returns: String

ExtractionServiceBuilder

Builder for composing an extraction service with Tower middleware layers.

Layers are applied in the order: Tracing → Metrics → Timeout → ConcurrencyLimit → Service.

Methods

default()

Signature:

pub fn default() -> ExtractionServiceBuilder

Example:

let result = ExtractionServiceBuilder::default();

Returns: ExtractionServiceBuilder

ExtractionSummary

Summary for a unified extraction call.

Field	Type	Default	Description
`inputs`	`usize`	—	Number of inputs submitted by the caller.
`results`	`usize`	—	Number of extraction results produced.
`errors`	`usize`	—	Number of per-input errors.
`remote_urls`	`usize`	—	Number of URI inputs that resolved to remote HTTP(S) URLs.
`pages_crawled`	`usize`	—	Number of HTML pages crawled or scraped.
`documents_downloaded`	`usize`	—	Number of downloaded non-HTML documents extracted from URLs.

FictionBookExtractor

FictionBook document extractor.

Supports FictionBook 2.0 format with proper section hierarchy and inline formatting.

Methods

default()

Signature:

pub fn default() -> FictionBookExtractor

Example:

let result = FictionBookExtractor::default();

Returns: FictionBookExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

FictionBookMetadata

FictionBook (FB2) metadata.

Field	Type	Default	Description
`genres`	`Vec<String>`	`vec![]`	Genre tags as declared in the FB2 `<genre>` elements.
`sequences`	`Vec<String>`	`vec![]`	Book series (sequence) names, if any.
`annotation`	`Option<String>`	`Default::default()`	Short annotation / summary from the FB2 `<annotation>` element.

FileExtractionConfig

Per-file extraction configuration overrides for batch processing.

All fields are Option<T> — None means “use the batch-level default.” This type is used by config and extract_batch to allow heterogeneous extraction settings within a single batch.

Excluded Fields

The following ExtractionConfig fields are batch-level only and cannot be overridden per file:

max_concurrent_extractions — controls batch parallelism
use_cache — global caching policy
acceleration — shared ONNX execution provider
security_limits — global archive security policy

Field	Type	Default	Description
`mime_detection_policy`	`Option<MimeDetectionPolicy>`	`Default::default()`	Override MIME inference policy for this file.
`enable_quality_processing`	`Option<bool>`	`Default::default()`	Override quality post-processing for this file.
`ocr`	`Option<OcrConfig>`	`Default::default()`	Override OCR configuration for this file (None in the Option = use batch default).
`force_ocr`	`Option<bool>`	`Default::default()`	Override force OCR for this file.
`ocr_strategy`	`Option<OcrStrategy>`	`Default::default()`	Override the OCR page-selection strategy for this file.
`force_ocr_pages`	`Option<Vec<u32>>`	`vec![]`	Override force OCR pages for this file (1-indexed page numbers).
`disable_ocr`	`Option<bool>`	`Default::default()`	Override disable OCR for this file.
`chunking`	`Option<ChunkingConfig>`	`Default::default()`	Override chunking configuration for this file.
`content_filter`	`Option<ContentFilterConfig>`	`Default::default()`	Override content filtering configuration for this file.
`images`	`Option<ImageExtractionConfig>`	`Default::default()`	Override image extraction configuration for this file.
`pdf_options`	`Option<PdfConfig>`	`Default::default()`	Override PDF options for this file.
`token_reduction`	`Option<TokenReductionOptions>`	`Default::default()`	Override token reduction for this file.
`language_detection`	`Option<LanguageDetectionConfig>`	`Default::default()`	Override language detection for this file.
`pages`	`Option<PageConfig>`	`Default::default()`	Override page extraction for this file.
`keywords`	`Option<KeywordConfig>`	`Default::default()`	Override keyword extraction for this file.
`postprocessor`	`Option<PostProcessorConfig>`	`Default::default()`	Override post-processor for this file.
`html_options`	`Option<ConversionOptions>`	`Default::default()`	Override HTML conversion options for this file.
`html_output`	`Option<HtmlOutputConfig>`	`Default::default()`	Override styled HTML output configuration for this file.
`result_format`	`Option<ResultFormat>`	`Default::default()`	Override result format for this file.
`output_format`	`Option<OutputFormat>`	`Default::default()`	Override output content format for this file.
`include_document_structure`	`Option<bool>`	`Default::default()`	Override document structure output for this file.
`layout`	`Option<LayoutDetectionConfig>`	`Default::default()`	Override layout detection for this file.
`transcription`	`Option<TranscriptionConfig>`	`Default::default()`	Transcription configuration (see ExtractionConfig for docs).
`timeout_secs`	`Option<u64>`	`Default::default()`	Override per-file extraction timeout in seconds. When set, the extraction for this file will be canceled after the specified duration. A timed-out file produces an error result without affecting other files in the batch.
`tree_sitter`	`Option<TreeSitterConfig>`	`Default::default()`	Override tree-sitter configuration for this file.
`structured_extraction`	`Option<StructuredExtractionConfig>`	`Default::default()`	Override structured extraction configuration for this file. When set, enables LLM-based structured extraction with a JSON schema for this specific file. The extracted content is sent to a VLM/LLM and the response is parsed according to the provided schema.
`url`	`Option<UrlExtractionConfig>`	`Default::default()`	Override URL ingestion and crawl configuration for this file.
`ner`	`Option<NerConfig>`	`Default::default()`	Override named-entity recognition configuration for this file.
`redaction`	`Option<RedactionConfig>`	`Default::default()`	Override redaction configuration for this file.
`summarization`	`Option<SummarizationConfig>`	`Default::default()`	Override summarization configuration for this file.
`translation`	`Option<TranslationConfig>`	`Default::default()`	Override translation configuration for this file.
`page_classification`	`Option<PageClassificationConfig>`	`Default::default()`	Override per-page classification configuration for this file.
`chunk_classification`	`Option<ChunkClassificationConfig>`	`Default::default()`	Override per-chunk classification configuration for this file.
`captioning`	`Option<CaptioningConfig>`	`Default::default()`	Override VLM captioning configuration for this file.
`qr_codes`	`Option<bool>`	`Default::default()`	Override QR-code detection for this file.

FileHeader

The 256-byte file header at the start of every HWP 5.0 document.

Field	Type	Default	Description
`flags`	`u32`	—	Flags word from bytes 36–39 (little-endian u32): bit 0 = compressed, bit 1 = encrypted.

FontScheme

Font scheme containing major (heading) and minor (body) fonts.

Field	Type	Default	Description
`name`	`String`	—	Font scheme name.
`major_latin`	`Option<String>`	`Default::default()`	Major (heading) font - Latin script.
`major_east_asian`	`Option<String>`	`Default::default()`	Major (heading) font - East Asian script.
`major_complex_script`	`Option<String>`	`Default::default()`	Major (heading) font - Complex script.
`minor_latin`	`Option<String>`	`Default::default()`	Minor (body) font - Latin script.
`minor_east_asian`	`Option<String>`	`Default::default()`	Minor (body) font - East Asian script.
`minor_complex_script`	`Option<String>`	`Default::default()`	Minor (body) font - Complex script.

Footnote

Footnote in Djot.

Field	Type	Default	Description
`label`	`String`	—	Footnote label
`content`	`Vec<FormattedBlock>`	—	Footnote content blocks

FootnoteAnchor

A footnote anchor reference in markdown text.

Represents a [^label] use-site (not a definition).

Field	Type	Default	Description
`label`	`String`	—	The label of the footnote reference (e.g., “1” in `[^1]`).
`offset`	`usize`	—	Byte offset of the anchor in the markdown text.

FootnoteConfig

Configuration for markdown footnote and citation parsing.

Field	Type	Default	Description
`parse_citations`	`bool`	`true`	Whether to parse the structured citation block (default: true). When enabled, the parser will look for and extract citations from the block after `---` + `<!-- citations ... -->`.

Methods

default()

Signature:

pub fn default() -> FootnoteConfig

Example:

let result = FootnoteConfig::default();

Returns: FootnoteConfig

with_parse_citations()

Set whether to parse the citation block.

Signature:

pub fn with_parse_citations(self, enabled: bool) -> FootnoteConfig

Example:

let result = instance.with_parse_citations(true);

Parameters:

Name	Type	Required	Description
`enabled`	`bool`	Yes	The enabled

Returns: FootnoteConfig

FootnoteDefinition

A footnote definition from markdown text.

Represents [^label]: content declarations (including multi-line continuations).

Field	Type	Default	Description
`label`	`String`	—	The label of the footnote (e.g., “1” in `[^1]: ...`).
`content`	`String`	—	The full content of the footnote definition.
`offset`	`usize`	—	Byte offset of the definition line in the markdown text.

FormattedBlock

Block-level element in a Djot document.

Represents structural elements like headings, paragraphs, lists, code blocks, etc.

Field	Type	Default	Description
`block_type`	`BlockType`	—	Type of block element
`level`	`Option<usize>`	`None`	Heading level (1-6) for headings, or nesting level for lists
`inline_content`	`Vec<InlineElement>`	—	Inline content within the block
`attributes`	`Option<Attributes>`	`None`	Element attributes (classes, IDs, key-value pairs)
`language`	`Option<String>`	`None`	Language identifier for code blocks
`code`	`Option<String>`	`None`	Raw code content for code blocks
`children`	`Vec<FormattedBlock>`	`/* serde(default) */`	Nested blocks for containers (blockquotes, list items, divs)

Formula

A mathematical formula extracted from a document.

Three kinds of sources populate this type. Layout-guided OCR detects formula regions and recognizes them; those formulas carry a bbox and a page. VLM OCR recognizes formulas in transcribed text without layout, so its formulas carry no geometry. Markup extraction (DOCX, PPTX, ODT, EPUB, HTML, JATS, LaTeX, Markdown, and related formats) converts embedded math to LaTeX, also without geometry.

Field	Type	Default	Description
`latex`	`String`	—	LaTeX source of the formula, without surrounding `$$` delimiters. Markup converters and formula OCR produce real LaTeX. The native PDF layout path stores the plain text of a detected formula region, which keeps the original Unicode math characters instead of LaTeX commands. To render the formula in Markdown or other formats, wrap it in `$$..$$`.
`bbox`	`Option<BoundingBox>`	`/* serde(default) */`	Bounding box of the formula region on its page. `None` for markup sources. PDF OCR sources report PDF point coordinates with the origin at the bottom-left of the page, comparable to native PDF geometry. Image sources, and PDF pages whose geometry is unavailable, report pixels of the image the OCR backend saw. The C FFI reports an absent bbox as a null pointer.
`page`	`Option<u32>`	`/* serde(default) */`	1-indexed page number the formula appears on. `None` when the source format has no page concept. The C FFI reports an absent page as `0`.

FormulaModelPaths

Local filenames of the downloaded model set.

Field	Type	Default	Description
`resizer`	`PathBuf`	—	Resizer
`encoder`	`PathBuf`	—	Encoder
`decoder`	`PathBuf`	—	Decoder
`tokenizer`	`PathBuf`	—	Tokenizer

GenericCache

Thread-safe, filesystem-backed cache with automatic LRU-style eviction.

GenericCache stores opaque blobs keyed by a content-hash string. It tracks in-flight deletions to prevent read-during-delete races and periodically runs a background cleanup that enforces the configured age and size limits.

GeoJsonExtractionConfig

Since: v1.1

Configuration for GeoJSON extraction.

GeoJSON coordinates can dominate extraction output and duplicate large geometry payloads in rendered content and metadata. The default therefore emits a bounded aggregate summary. Set Self::include_full_coordinates only when callers need every coordinate in the extracted text and accept output proportional to the input.

Field	Type	Default	Description
`include_full_coordinates`	`bool`	—	Include every coordinate in rendered content and `flattened_fields` metadata. Defaults to `false`. When false, extraction reports feature, property, geometry, position-count, bounds, truncation, and discarded-category metadata and emits a `ProcessingWarning` for every GeoJSON input.

GlinerManifestEntry

A single GLiNER model artifact entry in the cache manifest.

Field	Type	Default	Description
`relative_path`	`String`	—	Relative path within the Hugging Face Hub cache directory.
`sha256`	`String`	—	Expected SHA-256 checksum from the pinned Hugging Face revision.
`size_bytes`	`u64`	—	Expected file size in bytes from the pinned Hugging Face revision.
`source_url`	`String`	—	Hugging Face source URL for downloading.

GlmOcrBackend

GLM-OCR backend using candle transformers.

A compact vision-language model (0.9 B) for full-page document parsing. Supports text recognition, tables, formulas, charts, and image captioning through a unified interface with markdown output.

Constructor notes

GlmOcrBackend::new(task, layout_mode) stores the default task and layout mode. dtype defaults to F32 — the only dtype validated during the smoke test. Use GlmOcrBackend::with_dtype to override.

Configuration

GLM-OCR accepts backend options for task, device, and layout mode selection:

{
  "task": "ocr",
  "device": "auto",
  "layout_mode": "whole_page",
  "enable_chart_understanding": false,
  "cache_dir": "/path/to/huggingface/cache"
}

task (string): "ocr" (default), "table", "formula", "chart", "caption"
device (string): "auto", "cpu", "cuda", "metal"
layout_mode (string): "whole_page" (default), "paired" (requires layout-detection feature)
enable_chart_understanding (boolean): route chart regions to chart understanding
cache_dir (string, optional): explicit Hugging Face Hub cache root

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

emits_structured_markdown()

Signature:

pub fn emits_structured_markdown(&self) -> bool

Example:

let result = instance.emits_structured_markdown();

Returns: bool

confidence_semantics()

GLM-OCR reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

GlmOcrBackendOptions

Runtime options accepted by the candle-glm-ocr backend.

Field	Type	Default	Description
`task`	`Option<GlmOcrTaskKind>`	`Default::default()`	Optional recognition task; the backend constructor’s task is used when absent.
`device`	`Option<CandleDevicePreference>`	`Default::default()`	Optional per-call device override.
`layout_mode`	`Option<GlmOcrLayoutMode>`	`Default::default()`	Optional whole-page or layout-paired dispatch mode.
`enable_chart_understanding`	`Option<bool>`	`Default::default()`	Whether chart regions use chart understanding instead of captioning.
`cache_dir`	`Option<String>`	`Default::default()`	Optional Hugging Face cache root.

GridCell

Individual grid cell with position and span metadata.

Field	Type	Default	Description
`content`	`String`	—	Cell text content.
`row`	`u32`	—	Zero-indexed row position.
`col`	`u32`	—	Zero-indexed column position.
`row_span`	`u32`	`1`	Number of rows this cell spans.
`col_span`	`u32`	`1`	Number of columns this cell spans.
`is_header`	`bool`	`/* serde(default) */`	Whether this is a header cell.
`bbox`	`Option<BoundingBox>`	`None`	Bounding box for this cell (if available).
`heading_level`	`Option<u8>`	`/* serde(default) */`	Outline level (1-6) of the heading style this cell’s text carries, when it has one. A DOCX banner row – row 0, one cell spanning the grid, styled `Heading1`..`Heading6` – is what Word’s navigation pane and a `TOC` field treat as the document’s outline, but as a table cell it reached consumers as anonymous text (GH#1587). `content` is deliberately left as the bare cell text: prefixing it with `#` would put a markdown heading inside a table cell, which is invalid where it lands and changes text every existing consumer already reads. This field is the signal instead, so a caller can decide for itself whether a `heading 2` in a banner row is a section title or a column label.
`style_name`	`Option<String>`	`/* serde(default) */`	Human-readable name of the paragraph style applied to this cell’s text (`heading 2`). Carries the style even when it resolves to no outline level, so a caller can key on a named style this library does not map to a heading. See `GridCell::heading_level`.

GzipExtractor

Gzip archive extractor.

Decompresses gzip files and extracts text content from the compressed data.

Methods

default()

Signature:

pub fn default() -> GzipExtractor

Example:

let result = GzipExtractor::default();

Returns: GzipExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

HeaderMetadata

Header/heading element metadata.

Field	Type	Default	Description
`level`	`u8`	—	Header level: 1 (h1) through 6 (h6)
`text`	`String`	—	Normalized text content of the header
`id`	`Option<String>`	`None`	HTML id attribute if present
`depth`	`u32`	—	Document tree depth at the header element
`html_offset`	`u32`	—	Byte offset in original HTML document

HeadingContext

Heading context for a chunk within a Markdown document.

Contains the heading hierarchy from document root to this chunk’s section.

Field	Type	Default	Description
`headings`	`Vec<HeadingLevel>`	—	The heading hierarchy from document root to this chunk’s section. Index 0 is the outermost (h1), last element is the most specific.

HeadingLevel

A single heading in the hierarchy.

Field	Type	Default	Description
`level`	`u8`	—	Heading depth (1 = h1, 2 = h2, etc.)
`text`	`String`	—	The text content of the heading.

HealthResponse

Health check response.

Field	Type	Default	Description
`status`	`String`	—	Health status
`version`	`String`	—	API version
`plugins`	`Option<PluginStatus>`	`None`	Plugin status (optional)

HeuristicsConfig

Configuration for document chunking and analysis heuristics.

Every threshold is a public field so callers can override any subset via struct-update syntax: HeuristicsConfig { text_layer_threshold: 0.5, ..the default constructor }.

Field	Type	Default	Description
`enable_pdf_text_heuristics`	`bool`	`true`	Enable PDF text-layer detection heuristics. When `true`, PDFs with a substantial text layer will skip chunking. Default: `true`.
`text_layer_threshold`	`f32`	`0.7`	Minimum fraction of pages that must have text to skip chunking. Range `0.0..=1.0`. Default: `0.7` (70 % of pages).
`file_size_threshold_bytes`	`u64`	`10485760`	File size threshold in bytes for considering chunking. Files smaller than this are processed without chunking. Default: 10 MiB (10 × 1 024 × 1 024).
`page_count_threshold`	`u32`	`50`	Page count threshold for considering chunking. Documents with fewer pages are processed without chunking. Default: 50.
`target_pages_per_chunk`	`u32`	`10`	Target number of pages per chunk for optimal parallel processing. Default: 10.
`max_pages_per_chunk`	`u32`	`25`	Hard cap on pages per chunk. No chunk will exceed this limit. Must be ≥ `target_pages_per_chunk`. Default: 25.
`disk_processing_threshold_bytes`	`u64`	`52428800`	File size threshold for disk-based processing. Files larger than this are buffered to disk to prevent OOM. Default: 50 MiB (50 × 1 024 × 1 024).
`min_chars_per_page`	`u32`	`50`	Minimum characters per page to consider a page as having text. Default: 50.
`max_xlsx_sheet_count`	`u32`	`200`	Maximum sheet count allowed in an XLSX workbook. Workbooks beyond this are rejected pre-extraction to avoid OOM / abusive billing inflation. Default: 200.
`max_xlsx_workbook_cells`	`u64`	`5000000`	Maximum cell count (sheets × rows × columns approximation) in an XLSX workbook. Default: 5 000 000 (≈ 200 sheets × 25 k cells).
`max_pptx_embedded_count`	`u32`	`50`	Maximum number of OLE-embedded objects extractable from a single PPTX or DOCX. Protects against zip-bomb-style nested-document abuse. Default: 50.

Methods

default()

Signature:

pub fn default() -> HeuristicsConfig

Example:

let result = HeuristicsConfig::default();

Returns: HeuristicsConfig

validate()

Validate the configuration.

Errors:

Returns HeuristicsError::ConfigError when:

target_pages_per_chunk is 0
max_pages_per_chunk < target_pages_per_chunk
file_size_threshold_bytes is 0

Signature:

pub fn validate(&self) -> Result<(), XbergError>

Example:

instance.validate()?;

Returns: No return value.

Errors: Returns Err(XbergError).

HierarchicalBlock

Field	Type	Default	Description
`text`	`String`	—	The text content of this block
`font_size`	`f32`	—	The font size of the text in this block
`level`	`String`	—	The hierarchy level of this block (H1-H6 or Body) Levels correspond to HTML heading tags: - “h1”: Top-level heading - “h2”: Secondary heading - “h3”: Tertiary heading - “h4”: Quaternary heading - “h5”: Quinary heading - “h6”: Senary heading - “body”: Body text (no heading level)
`bbox`	`Option<HierarchicalBoundingBox>`	`None`	Bounding box information for the block Contains left, top, right, and bottom coordinates in PDF units.

HierarchicalBoundingBox

A text block with hierarchy level assignment.

Represents a block of text with semantic heading information extracted from font size clustering and hierarchical analysis.

Field	Type	Default	Description
`left`	`f32`	—	Left coordinate.
`top`	`f32`	—	Top coordinate.
`right`	`f32`	—	Right coordinate.
`bottom`	`f32`	—	Bottom coordinate.

HierarchyBoundingBox

A bounding box for text or elements.

Field	Type	Default	Description
`left`	`f32`	—	Left x-coordinate
`top`	`f32`	—	Top y-coordinate
`right`	`f32`	—	Right x-coordinate
`bottom`	`f32`	—	Bottom y-coordinate

HierarchyConfig

Hierarchy extraction configuration for PDF text structure analysis.

Enables extraction of document hierarchy levels (H1-H6) based on font size clustering and semantic analysis. When enabled, hierarchical blocks are included in page content.

Field	Type	Default	Description
`enabled`	`bool`	`true`	Enable hierarchy extraction
`k_clusters`	`usize`	`3`	Number of font size clusters to use for hierarchy levels (1-7) Default: 3, which provides two heading levels plus body text. Larger values create more fine-grained hierarchy levels.
`include_bbox`	`bool`	`true`	Include bounding box information in hierarchy blocks

Methods

default()

Signature:

pub fn default() -> HierarchyConfig

Example:

let result = HierarchyConfig::default();

Returns: HierarchyConfig

HocrWord

Represents a word extracted from hOCR (or any source) with position and confidence information.

Field	Type	Default	Description
`text`	`String`	—	Recognized word text.
`left`	`u32`	—	Left edge of the word bounding box in pixels.
`top`	`u32`	—	Top edge of the word bounding box in pixels.
`width`	`u32`	—	Bounding box width in pixels.
`height`	`u32`	—	Bounding box height in pixels.
`confidence`	`f64`	—	OCR confidence score (0.0–100.0).

HtmlExtractionResult

Result of HTML extraction with optional images and warnings.

Field	Type	Default	Description
`markdown`	`String`	—	Extracted content as Markdown.
`images`	`Vec<ExtractedInlineImage>`	—	Inline images extracted from the HTML document.
`warnings`	`Vec<String>`	—	Non-fatal warnings generated during extraction (e.g. unsupported elements).

HtmlExtractor

HTML document extractor using html-to-markdown.

Methods

default()

Signature:

pub fn default() -> HtmlExtractor

Example:

let result = HtmlExtractor::default();

Returns: HtmlExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_sync()

Signature:

pub fn extract_sync(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_sync(b"data", "value", &todo!())?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

HtmlMetadata

HTML metadata extracted from HTML documents.

Includes document-level metadata, Open Graph data, Twitter Card metadata, and extracted structural elements (headers, links, images, structured data).

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Document title from `<title>` tag
`description`	`Option<String>`	`Default::default()`	Document description from `<meta name="description">` tag
`keywords`	`Vec<String>`	`vec![]`	Document keywords from `<meta name="keywords">` tag, split on commas
`author`	`Option<String>`	`Default::default()`	Document author from `<meta name="author">` tag
`canonical_url`	`Option<String>`	`Default::default()`	Canonical URL from `<link rel="canonical">` tag
`base_href`	`Option<String>`	`Default::default()`	Base URL from `<base href="">` tag for resolving relative URLs
`language`	`Option<String>`	`Default::default()`	Document language from `lang` attribute
`text_direction`	`Option<TextDirection>`	`Default::default()`	Document text direction from `dir` attribute
`open_graph`	`HashMap<String, String>`	`HashMap::new()`	Open Graph metadata (og:* properties) for social media Keys like “title”, “description”, “image”, “url”, etc.
`twitter_card`	`HashMap<String, String>`	`HashMap::new()`	Twitter Card metadata (twitter:* properties) Keys like “card”, “site”, “creator”, “title”, “description”, “image”, etc.
`meta_tags`	`HashMap<String, String>`	`HashMap::new()`	Additional meta tags not covered by specific fields Keys are meta name/property attributes, values are content
`headers`	`Vec<HeaderMetadata>`	`vec![]`	Extracted header elements with hierarchy
`links`	`Vec<LinkMetadata>`	`vec![]`	Extracted hyperlinks with type classification
`images`	`Vec<ImageMetadataType>`	`vec![]`	Extracted images with source and dimensions
`structured_data`	`Vec<StructuredData>`	`vec![]`	Extracted structured data blocks

HtmlOutputConfig

Configuration for styled HTML output.

When set on html_output alongside output_format = OutputFormat::Html, the pipeline builds a StyledHtmlRenderer instead of the plain comrak-based renderer.

Field	Type	Default	Description
`css`	`Option<String>`	`None`	Inline CSS string injected into the output after the theme stylesheet. Concatenated after `css_file` content when both are set.
`css_file`	`Option<PathBuf>`	`None`	Path to a CSS file loaded once at renderer construction time. Concatenated before `css` when both are set.
`theme`	`HtmlTheme`	`HtmlTheme::Unstyled`	Built-in colour/typography theme. Default: `HtmlTheme::Unstyled`.
`class_prefix`	`String`	`"kb-"`	CSS class prefix applied to every emitted class name. Default: `"kb-"`. Change this if your host application already uses classes that start with `kb-`.
`embed_css`	`bool`	`true`	When `true` (default), write the resolved CSS into a `<style>` block immediately after the opening `<div class="{prefix}doc">`. Set to `false` to emit only the structural markup and wire up your own stylesheet targeting the `kb-*` class names.

Methods

default()

Signature:

pub fn default() -> HtmlOutputConfig

Example:

let result = HtmlOutputConfig::default();

Returns: HtmlOutputConfig

HwpDocument

An extracted HWP document, consisting of one or more body-text sections.

Field	Type	Default	Description
`sections`	`Vec<Section>`	`vec![]`	All sections from all BodyText/SectionN streams.
`char_shapes`	`Vec<CharShape>`	`vec![]`	Global character shape table from DocInfo.
`images`	`Vec<HwpImage>`	`vec![]`	Extracted images from BinData.
`summary_info`	`Option<SummaryInfo>`	`Default::default()`	Document metadata from the `\x05HwpSummaryInformation` OLE property-set stream, or `None` if the stream was absent or unparsable (#105).
`warnings`	`Vec<String>`	`vec![]`	Human-readable descriptions of content that could not be fully parsed (truncated body-text streams, unsupported nested tables, etc.) — surfaced by the extractor as `ProcessingWarning`s instead of silently dropping the data (#236). Each entry names what was lost, per the extractor warning convention (`crate::core::diagnostics`).

HwpExtractor

Extractor for Hangul Word Processor (.hwp) files.

Supports HWP 5.0 format, the standard document format in South Korea.

Methods

default()

Signature:

pub fn default() -> HwpExtractor

Example:

let result = HwpExtractor::default();

Returns: HwpExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

HwpImage

A raw image blob extracted from a BinData stream in an HWP document.

Field	Type	Default	Description
`name`	`String`	—	Stream path used as a stable identifier (e.g. `"BinData/BIN0001.jpg"`).
`data`	`Vec<u8>`	—	Raw image bytes as stored in the BinData stream.

HwpTable

A table extracted from a HWPTAG_TABLE record and its nested HWPTAG_LIST_HEADER cells.

Field	Type	Default	Description
`rows`	`Vec<Vec<String>>`	`vec![]`	Cell text, as a row-major grid (`rows[r][c]`). Cells covered by a merged (rowspan/colspan > 1) neighbor are empty strings rather than omitted, so every row has `column_count` entries.

HwpxExtractor

Extractor for Hangul Word Processor XML (.hwpx) files.

Supports HWPX (Open HWPML), the ZIP-based XML successor to the binary HWP 5.0 format.

Methods

default()

Signature:

pub fn default() -> HwpxExtractor

Example:

let result = HwpxExtractor::default();

Returns: HwpxExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

ImageClassifyInput

Grouped arguments for classify.

Not FFI-exposed; grouping simply keeps classify under the workspace’s six-parameter limit. alef(skip) on classify does NOT propagate to this struct – alef generates a binding type for any reachable pub struct – so the skip has to be repeated here, or every language package grows an ImageClassifyInput for a purely internal parameter bag. It stays pub rather than pub(crate) because tests/image_classification.rs is a separate crate and constructs it.

Field	Type	Default	Description
`bytes`	`Vec<u8>`	—	Raw image bytes (should be decodable to standard formats).
`format`	`String`	—	Image format (e.g., “jpeg”, “png”, “ccitt”).
`width`	`Option<u32>`	`None`	Image width in pixels.
`height`	`Option<u32>`	`None`	Image height in pixels.
`colorspace`	`Option<String>`	`None`	Colorspace name (e.g., “RGB”, “CMYK”, “Gray”, “Indexed”).
`bits_per_component`	`Option<u32>`	`None`	Bits per color component (e.g., 1, 8, 16).
`is_mask`	`bool`	—	Whether this image is a transparency or alpha mask.

ImageDimensions

Image dimensions in pixels.

Field	Type	Default	Description
`width`	`u32`	—	Width in pixels.
`height`	`u32`	—	Height in pixels.

ImageDpi

Horizontal and vertical image resolution in dots per inch.

Field	Type	Default	Description
`horizontal`	`f64`	—	Horizontal resolution.
`vertical`	`f64`	—	Vertical resolution.

ImageDpiConfig

Image extraction DPI configuration (internal use).

Note: This is an internal type used for image preprocessing. For the main extraction configuration, see ExtractionConfig.

Field	Type	Default	Description
`target_dpi`	`i32`	`300`	Target DPI for image normalization
`max_image_dimension`	`i32`	`4096`	Maximum image dimension (width or height)
`auto_adjust_dpi`	`bool`	`true`	Whether to auto-adjust DPI based on content
`min_dpi`	`i32`	`72`	Minimum DPI threshold
`max_dpi`	`i32`	`600`	Maximum DPI threshold

Methods

default()

Signature:

pub fn default() -> ImageDpiConfig

Example:

let result = ImageDpiConfig::default();

Returns: ImageDpiConfig

ImageExtractionConfig

Image extraction configuration.

Field	Type	Default	Description
`extract_images`	`bool`	`true`	Extract images from documents
`target_dpi`	`i32`	`300`	Target DPI for image normalization
`max_image_dimension`	`i32`	`4096`	Maximum dimension for images (width or height)
`inject_placeholders`	`bool`	`true`	Whether to inject image reference placeholders into markdown output. When `true` (default), image references like `!Image 1` are appended to the markdown. Set to `false` to extract images as data without polluting the markdown output.
`auto_adjust_dpi`	`bool`	`true`	Automatically adjust DPI based on image content
`min_dpi`	`i32`	`72`	Minimum DPI threshold
`max_dpi`	`i32`	`600`	Maximum DPI threshold
`max_images_per_page`	`Option<u32>`	`None`	Maximum number of image objects to extract per PDF page. Some PDFs (e.g. technical diagrams stored as thousands of raster fragments) can trigger extremely long or indefinite extraction times when every image object on a dense page is decoded individually via the PDF extractor. Setting this limit causes xberg to stop collecting individual images once the count per page reaches the cap and emit a warning instead. `None` (default) means no limit — all images are extracted.
`classify`	`bool`	`false`	When `true`, extracted images are classified by kind and grouped into clusters where they appear to belong to one figure. Defaults to `false` — opt in explicitly to avoid unexpected ML overhead.
`include_page_rasters`	`bool`	`false`	When `true`, full-page renders produced during OCR preprocessing are captured and returned as `ImageKind::PageRaster` entries in `ExtractedDocument.images`. PDF + OCR only. No rasters are captured for non-PDF inputs or when the document-level OCR bypass is active (whole-document backend). When OCR is enabled and this flag is set but the active backend skips per-page rendering, a `ProcessingWarning` is emitted in `ExtractedDocument.processing_warnings`. Defaults to `false`. Enable when downstream consumers need page thumbnails (e.g. citation previews, visual grounding).
`run_ocr_on_images`	`bool`	`true`	Run OCR on extracted images and include the recognized text in the document content. When `true` (default) and `ExtractionConfig.ocr` is configured, extracted images are processed with the configured OCR backend. Set to `false` to extract images without OCR processing, even when OCR is enabled.
`ocr_text_only`	`bool`	`false`	When `true`, image OCR results are rendered as plain text without the `![...](...)` markdown placeholder. Only takes effect when `run_ocr_on_images` is also `true`.
`append_ocr_text`	`bool`	`true`	Append the OCR text of an image after its placeholder in Markdown and HTML output. When `true` (default), the text that `run_ocr_on_images` recognizes follows the placeholder. Set to `false` to show the placeholder alone. Plain and Djot output hold the text with either value. Has no effect when `ocr_text_only` is `true`.
`output_format`	`ImageOutputFormat`	`ImageOutputFormat::Native`	Target format for re-encoding extracted images. When set to anything other than `Native`, each extracted image is re-encoded to the requested format before being returned. This lets callers receive uniform output without duplicating encode logic downstream. Defaults to `Native` — no re-encode pass is performed and `ExtractedImage.format` reflects the source extractor’s output.
`svg`	`SvgOptions`	—	SVG-specific knobs for the image-encode pipeline. Controls sanitization and rasterization DPI when the source or output format is SVG. Only available when the `svg` feature is active.
`include_data_base64`	`bool`	`false`	When `true`, populate `ExtractedImage::data_base64` with a Base64-encoded copy of the raw image bytes. Useful for JSON-only clients that cannot efficiently parse the default integer-array serialization of `data`. Defaults to `false`; enabling it doubles the in-memory image representation for the duration of the response.

Methods

default()

Signature:

pub fn default() -> ImageExtractionConfig

Example:

let result = ImageExtractionConfig::default();

Returns: ImageExtractionConfig

ImageExtractor

Image extractor for various image formats.

Supports: PNG, JPEG, WebP, BMP, TIFF, GIF. Extracts dimensions, format, and EXIF metadata. Optionally runs OCR when configured. When layout detection is also enabled, uses per-region OCR with markdown formatting based on detected layout classes.

Methods

default()

Signature:

pub fn default() -> ImageExtractor

Example:

let result = ImageExtractor::default();

Returns: ImageExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

ImageMetadata

Image metadata extracted from image files.

Includes dimensions, format, and EXIF data.

Field	Type	Default	Description
`width`	`u32`	—	Image width in pixels
`height`	`u32`	—	Image height in pixels
`format`	`String`	—	Image format (e.g., “PNG”, “JPEG”, “TIFF”)
`exif`	`HashMap<String, String>`	`HashMap::new()`	EXIF metadata tags

ImageMetadataType

Image element metadata.

Field	Type	Default	Description
`src`	`String`	—	Image source (URL, data URI, or SVG content)
`alt`	`Option<String>`	`None`	Alternative text from alt attribute
`title`	`Option<String>`	`None`	Title attribute
`dimensions`	`Option<ImageDimensions>`	`None`	Image dimensions if available.
`image_type`	`ImageType`	—	Image type classification
`attributes`	`Vec<KeyValueAttribute>`	—	Additional attributes as key-value pairs.

ImageOcrResult

Result of OCR extraction from an image with optional page tracking.

Field	Type	Default	Description
`content`	`String`	—	Extracted text content
`boundaries`	`Option<Vec<PageBoundary>>`	`None`	Character byte boundaries per frame (for multi-frame TIFFs)
`page_contents`	`Option<Vec<PageContent>>`	`None`	Per-frame content information

ImagePreprocessingConfig

Image preprocessing configuration for OCR.

These settings control how images are preprocessed before OCR to improve text recognition quality. Different preprocessing strategies work better for different document types.

Field	Type	Default	Description
`target_dpi`	`i32`	`300`	Target DPI for the image (300 is standard, 600 for small text). For a PDF page, this resamples the already-rendered raster; it does not make the page re-render at a higher native resolution. A value above what `image::preprocessing::calculate_target_dpi`’s memory/dimension clamp allows is silently capped (GH#1786: 400 and 600 both clamped to the same ~372 on a Letter page and produced byte-identical output), so upscaling interpolated pixels this way adds no detail. To change the actual PDF render resolution, set `images.target_dpi` on `ExtractionConfig` instead (`image::dpi::effective_pdf_render_dpi`).
`auto_rotate`	`bool`	`false`	Auto-detect and correct image rotation.
`deskew`	`bool`	`true`	Correct skew (tilted images). Must be `false` with `none` or `off` binarization.
`denoise`	`bool`	`false`	Remove noise from the image.
`contrast_enhance`	`bool`	`false`	Enhance contrast for better text visibility. With `none` or `off` binarization, this applies background normalization and sharpening while preserving grayscale pixels.
`binarization_method`	`String`	`"otsu"`	Binarization method: “none” (alias “off”), “otsu”, “sauvola”, or “adaptive”. `deskew` must be `false` when this is `none` or `off`.
`invert_colors`	`bool`	`false`	Invert colors (white text on black → black on white).
`normalize_shaded_rows`	`bool`	`false`	Normalize shaded table rows (e.g. a subtotal row on a light or dark fill) before binarization, so each shaded band is stretched to its own dark-text-on-white polarity instead of being lost to a single whole-page threshold (GH#1785). This is a per-band step, not a replacement for `binarization_method`: no single whole-page method recovers every fill color, and the per-band step itself can regress a row style it does not fully model (e.g. a mid-grey fill with white text), so it defaults to `false` rather than being enabled unconditionally. Has no effect when `binarization_method` is `"none"` or `"off"` and `contrast_enhance` is `true`: that path re-reads the original image to apply background normalization and sharpening, discarding the per-band result. A `WARN` is emitted when the option is requested in that combination (GH#1837).

Methods

default()

Signature:

pub fn default() -> ImagePreprocessingConfig

Example:

let result = ImagePreprocessingConfig::default();

Returns: ImagePreprocessingConfig

ImagePreprocessingMetadata

Field	Type	Default	Description
`original_dimensions`	`PixelDimensions`	—	Original image dimensions in pixels.
`original_dpi`	`ImageDpi`	—	Original image resolution.
`target_dpi`	`i32`	—	Target DPI from configuration
`scale_factor`	`f64`	—	Scaling factor applied to the image
`auto_adjusted`	`bool`	—	Whether DPI was auto-adjusted based on content
`final_dpi`	`i32`	—	Final DPI after processing
`new_dimensions`	`Option<PixelDimensions>`	`None`	New dimensions after resizing (if resized).
`resample_method`	`String`	—	Resampling algorithm used (“LANCZOS3”, “CATMULLROM”, etc.)
`dimension_clamped`	`bool`	—	Whether dimensions were clamped to max_image_dimension
`calculated_dpi`	`Option<i32>`	`None`	Calculated optimal DPI (if auto_adjust_dpi enabled)
`skipped_resize`	`bool`	—	Whether resize was skipped (dimensions already optimal)
`resize_error`	`Option<String>`	`None`	Error message if resize failed

InfoResponse

Server information response.

Field	Type	Default	Description
`version`	`String`	—	API version
`rust_backend`	`bool`	—	Whether using Rust backend

InlineElement

Inline element within a block.

Represents text with formatting, links, images, etc.

Field	Type	Default	Description
`element_type`	`InlineType`	—	Type of inline element
`content`	`String`	—	Text content
`attributes`	`Option<Attributes>`	`None`	Element attributes
`metadata`	`Option<HashMap<String, String>>`	`None`	Additional metadata (e.g., href for links, src/alt for images)

InternalDocument

The internal flat document representation.

All extractors output this structure. It is converted to the public ExtractedDocument and DocumentStructure in the pipeline.

Implements Serialize/Deserialize so that foreign-language plugin implementations (Python, TypeScript, Ruby, etc.) can construct and return this type via JSON at the FFI/trait-bridge boundary.

Field	Type	Default	Description
`elements`	`Vec<InternalElement>`	`vec![]`	All elements in reading order. Append-only during extraction.
`relationships`	`Vec<Relationship>`	`vec![]`	Relationships between elements (source index → target). Stored separately from elements for cache-friendly iteration.
`source_format`	`String`	—	Source format identifier (e.g., “pdf”, “docx”, “html”, “markdown”).
`metadata`	`Metadata`	—	Document-level metadata (title, author, dates, etc.).
`images`	`Vec<ExtractedImage>`	`vec![]`	Extracted images (binary data). Referenced by index from `ElementKind::Image`.
`tables`	`Vec<Table>`	`vec![]`	Extracted tables (structured data). Referenced by index from `ElementKind::Table`.
`diagrams`	`Vec<DiagramGraph>`	`vec![]`	Node/edge graphs recovered from vector diagrams in the source. Populated by extractors that can read diagram geometry deterministically (SVG today). The `dot` renderer turns these into Graphviz DOT; every other renderer ignores them, so the field only ever adds output.
`uris`	`Vec<ExtractedUri>`	`vec![]`	URIs/links discovered during extraction (hyperlinks, image refs, citations, etc.).
`uris_dropped`	`usize`	—	Number of URIs `InternalDocument::push_uri` refused because `uris` was already at the per-document `MAX_URIS` cap. Not part of the plugin-bridge wire format: a foreign plugin deserializes straight into `uris` and never goes through the cap, so it has nothing to report here. `derive_extraction_result` turns a non-zero count into a `ProcessingWarning` (#76).
`children`	`Option<Vec<ArchiveEntry>>`	`vec![]`	Archive children: fully-extracted results for files within an archive. Only populated by archive extractors (ZIP, TAR, 7z, GZIP) when recursive extraction is enabled. Each entry contains the full `ExtractedDocument` for a child file that was extracted through the public pipeline.
`mime_type`	`String`	—	MIME type of the source document (e.g., “application/pdf”, “text/html”).
`processing_warnings`	`Vec<ProcessingWarning>`	`vec![]`	Non-fatal warnings collected during extraction.
`ocr_page_failures`	`Vec<OcrPageFailure>`	`vec![]`	Structured OCR backend failures collected while extracting PDF pages.
`annotations`	`Option<Vec<PdfAnnotation>>`	`vec![]`	PDF annotations (links, highlights, notes).
`prebuilt_pages`	`Option<Vec<PageContent>>`	`vec![]`	Pre-built per-page content (set by extractors that track page boundaries natively). When populated, `derive_extraction_result` uses this directly instead of attempting to reconstruct pages from element-level page numbers.
`pre_rendered_content`	`Option<String>`	`Default::default()`	Pre-rendered formatted content produced by the extractor itself. When an extractor has direct access to high-quality formatted output (e.g., html-to-markdown produces GFM markdown), it can store that here to bypass the lossy InternalDocument → renderer round-trip. `derive_extraction_result` will use this directly when the requested output format matches `metadata.output_format`.
`prebuilt_ocr_elements`	`Option<Vec<OcrElement>>`	`vec![]`	Pre-built OCR element list (set by extractors that have direct access to bounding-box element data alongside a separately produced coherent text). When populated, `derive_extraction_result` uses this directly instead of reconstructing `OcrElement`s from `OcrText` `InternalElement`s. This lets the image extractor carry Tesseract/paddle-ocr bounding-box metadata without injecting raw word tokens into the element list (which would otherwise corrupt `render_plain` and page content — issue #706).
`llm_usage`	`Option<Vec<LlmUsage>>`	`vec![]`	LLM usage records accumulated during extraction (e.g., VLM OCR per page). Populated by extractors that call LLM-backed backends (VLM OCR). `derive_extraction_result` transfers this to `ExtractedDocument.llm_usage`.
`revisions`	`Option<Vec<DocumentRevision>>`	`vec![]`	Track-changes revisions embedded in the source document. Set by format-specific extractors (DOCX, ODT, …) that parse change-tracking markup. `derive_extraction_result` transfers this directly to `ExtractedDocument.revisions`.
`form_fields`	`Vec<PdfFormField>`	`vec![]`	PDF form fields extracted from AcroForm or XFA-based forms. Set by the PDF extractor when `pdf_options.extract_form_fields = true`. `derive_extraction_result` transfers this directly to `ExtractedDocument.form_fields`.
`formulas`	`Vec<Formula>`	`vec![]`	Mathematical formulas recognized during layout-guided OCR. Set by the OCR pipeline (per-page formulas, renumbered to document pages). `derive_extraction_result` transfers this directly to `ExtractedDocument.formulas`.
`recorded_formulas`	`Vec<Formula>`	`vec![]`	Formulas that stay inside the text they came from. A table cell keeps its equation in the cell, and an HWP sentence keeps its equation in the sentence, so neither can become an element without changing that text. They are still formulas of the document. Kept apart from `formulas` because that field holds what the OCR pipeline produced, and `derive_extraction_result` treats an OCR entry as a second representation of an element. An entry here is not.
`ocr_text_only`	`bool`	—	When `true`, image OCR results are rendered as plain text without the `![...](...)` markdown placeholder. Set by the pipeline from `ImageExtractionConfig.ocr_text_only`.
`append_ocr_text`	`bool`	—	When `true` and `ocr_text_only` is `false`, append the OCR text after the image placeholder in the rendered output. Set by the pipeline from `ImageExtractionConfig.append_ocr_text`.
`escape_markdown`	`bool`	—	When `true` (the default), Markdown rendering backslash-escapes CommonMark-significant characters (`_[]()*=-#`) so the output round-trips safely through a CommonMark parser. When `false`, those escapes are stripped so prose reads identically to the already-unescaped text used in table cells. Set by the pipeline from `ExtractionConfig::escape_markdown`.
`include_watermarks`	`bool`	—	When `true`, Markdown rendering preserves watermark text such as arXiv identifiers. Set by the pipeline from `ContentFilterConfig::include_watermarks`.
`page_marker_format`	`Option<String>`	`Default::default()`	Page marker format (with `{page_num}` placeholder) when `PageConfig::insert_page_markers` is enabled, `None` otherwise. Set by the pipeline. Renderers use it to emit page markers verbatim instead of escaping or stripping them.
`table_anchors`	`bool`	—	When `true`, Markdown rendering inserts a `[TABLE:{table_id}]` marker immediately before each table’s rendered Markdown block. Set by the pipeline from `ExtractionConfig::table_anchors`. Defaults to `false`.
`ocr_coordinate_frame`	`Option<OcrPageCoordinateFrame>`	`Default::default()`	This OCR page’s authoritative processed-raster coordinate frame, captured before the page-local metadata it comes from is discarded (GH#1645). `#[serde(skip)]` for the same reason as the fields above – it never crosses the plugin-bridge JSON wire format – and is folded into `Metadata::additional` once the page document is final rather than becoming a new public binding type. `None` for every non-OCR document, and for an OCR’d page whose render raster was degenerate (0x0).
`ocr_text_claimed_by_tables`	`bool`	—	Set by an OCR backend when detected tables claimed every text element of this page (#1571), so the page text holds only lines the tables already carry. Never crosses the plugin-bridge JSON wire format.

InternalDocumentBuilder

Builder for constructing InternalDocument with an ergonomic push-based API.

Tracks nesting depth automatically for list and quote containers, and generates deterministic element IDs via blake3 hashing.

InternalDocumentExtractor

Low-level extraction capability used by native Rust extractors.

Native extractors implement this to produce the pipeline’s InternalDocument representation; the blanket impl below derives the public DocumentExtractor surface from it. Exposed for white-box tests and advanced consumers that need the pre-derivation document.

Skipped by the binding surface: it returns the internal-only InternalDocument, which is not part of the public binding API.

Methods

extract_content()

Extract an in-memory payload into the pipeline representation.

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Extract a path into the pipeline representation.

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Get the list of MIME types supported by this extractor.

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Get the priority of this extractor.

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

can_handle()

Check if this extractor can handle a specific path and MIME type.

Signature:

pub fn can_handle(&self, path: &PathBuf, mime_type: &str) -> bool

Example:

let result = instance.can_handle("value", "value");

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	The path
`mime_type`	`&str`	Yes	The mime type

Returns: bool

InternalElement

A single element in the internal flat document.

Elements are appended in reading order during extraction. The depth field and optional container markers enable tree reconstruction in the derivation step.

Field	Type	Default	Description
`id`	`InternalElementId`	—	Deterministic identifier.
`kind`	`ElementKind`	—	What kind of content this element represents.
`text`	`String`	—	Primary text content. Empty for non-text elements (images, page breaks).
`depth`	`u16`	—	Nesting depth (0 = root level). Extractors set this based on heading level, list indent, blockquote depth, etc. The tree derivation step uses depth changes to reconstruct parent-child relationships.
`page`	`Option<u32>`	`None`	Page number (1-indexed). `None` for non-paginated formats.
`bbox`	`Option<BoundingBox>`	`None`	Bounding box in document coordinates.
`layer`	`ContentLayer`	—	Content layer classification (Body, Header, Footer, Footnote).
`annotations`	`Vec<TextAnnotation>`	—	Inline annotations (formatting, links) on this element’s text content. Byte-range based, reuses the existing `TextAnnotation` type.
`attributes`	`Option<HashMap<String, String>>`	`None`	Format-specific key-value attributes. Used for CSS classes, LaTeX env names, slide layout names, etc.
`anchor`	`Option<String>`	`None`	Optional anchor/key for this element. Used by the relationship resolver to match references to targets. Examples: heading slug `"introduction"`, footnote label `"fn1"`, citation key `"smith2024"`, figure label `"fig:diagram"`.
`ocr_geometry`	`Option<OcrBoundingGeometry>`	`None`	OCR bounding geometry (rectangle or quadrilateral).
`ocr_confidence`	`Option<OcrConfidence>`	`None`	OCR confidence scores (detection + recognition).
`ocr_rotation`	`Option<OcrRotation>`	`None`	OCR rotation metadata.

InternalElementId

Deterministic element identifier, generated via blake3 hashing.

Format: "ie-{12 hex chars}" (48 bits from blake3, ~281 trillion address space). Same input always produces the same ID, enabling diffing and caching.

Serializes as a plain string ("ie-aabbccddeeff").

Methods

as_ref()

Signature:

pub fn as_ref(&self) -> String

Example:

let result = instance.as_ref();

Returns: String

InternedString

A reference to an interned string stored in an Arc.

This wraps an Arc<String> and provides convenient access to the string content. Multiple calls with the same string content will share the same Arc, reducing memory usage.

Methods

as_ref()

Signature:

pub fn as_ref(&self) -> String

Example:

let result = instance.as_ref();

Returns: String

JatsExtractor

JATS document extractor.

Supports JATS (Journal Article Tag Suite) XML documents in various versions, handling both the full article structure and minimal JATS subsets.

Methods

default()

Signature:

pub fn default() -> JatsExtractor

Example:

let result = JatsExtractor::default();

Returns: JatsExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

JatsMetadata

JATS (Journal Article Tag Suite) metadata.

Field	Type	Default	Description
`copyright`	`Option<String>`	`Default::default()`	Copyright statement from the article’s `<permissions>` element.
`license`	`Option<String>`	`Default::default()`	Open-access license URI from the article’s `<license>` element.
`history_dates`	`HashMap<String, String>`	`HashMap::new()`	Publication history dates keyed by event type (e.g. `"received"`, `"accepted"`).
`contributor_roles`	`Vec<ContributorRole>`	`vec![]`	Authors and contributors with their stated roles.

JsonExtractionConfig

Configuration options for JSON extraction behaviour.

Field	Type	Default	Description
`extract_schema`	`bool`	`false`	When `true`, a JSON schema summary is generated and stored in metadata.
`max_depth`	`usize`	`20`	Maximum object/array nesting depth before extraction stops recursing.
`array_item_limit`	`usize`	`500`	Maximum number of array items to process per array node.
`include_type_info`	`bool`	`false`	When `true`, field values are annotated with their JSON type.
`flatten_nested_objects`	`bool`	`true`	When `true`, nested objects are flattened into dot-separated key paths. When `false`, each nested object is retained as compact JSON under its parent path in `StructuredDataResult::flattened`.
`custom_text_field_patterns`	`Vec<String>`	`vec![]`	Additional field-name patterns (exact match, case-insensitive) treated as text fields.

Methods

default()

Signature:

pub fn default() -> JsonExtractionConfig

Example:

let result = JsonExtractionConfig::default();

Returns: JsonExtractionConfig

JupyterExtractor

Jupyter Notebook extractor.

Extracts content from Jupyter notebook JSON files, including:

Notebook metadata (kernel, language, nbformat version)
Cell content (code and markdown)
Cell outputs (text, HTML, etc.)
Cell-level metadata (tags, execution counts)

Methods

default()

Signature:

pub fn default() -> JupyterExtractor

Example:

let result = JupyterExtractor::default();

Returns: JupyterExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

KeyValueAttribute

A string key-value attribute.

Field	Type	Default	Description
`key`	`String`	—	Attribute name.
`value`	`String`	—	Attribute value.

KeynoteExtractor

Apple Keynote presentation extractor.

Supports .key files (modern iWork format, 2013+).

Extracts slide text and speaker notes from the IWA container: ZIP → Snappy → protobuf text fields.

Methods

default()

Signature:

pub fn default() -> KeynoteExtractor

Example:

let result = KeynoteExtractor::default();

Returns: KeynoteExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Keyword

Extracted keyword with metadata.

Field	Type	Default	Description
`text`	`String`	—	The keyword text.
`score`	`f32`	—	Relevance score (higher is better, algorithm-specific range).
`algorithm`	`KeywordAlgorithm`	—	Algorithm that extracted this keyword.
`positions`	`Option<Vec<usize>>`	`None`	Optional positions where keyword appears in text (character offsets).

KeywordConfig

Keyword extraction configuration.

Field	Type	Default	Description
`algorithm`	`KeywordAlgorithm`	`KeywordAlgorithm::Yake`	Algorithm to use for extraction.
`max_keywords`	`usize`	`10`	Maximum number of keywords to extract (default: 10).
`min_score`	`f32`	`0`	Minimum score threshold (0.0-1.0, default: 0.0). Keywords with scores below this threshold are filtered out. Note: Score ranges differ between algorithms.
`ngram_range`	`NgramRange`	—	N-gram range for keyword extraction (min, max). (1, 1) = unigrams only (1, 2) = unigrams and bigrams (1, 3) = unigrams, bigrams, and trigrams (default)
`language`	`Option<String>`	`"en"`	Language code for stopword filtering (e.g., “en”, “de”, “fr”). If None, no stopword filtering is applied.
`yake_params`	`Option<YakeParams>`	`None`	YAKE-specific tuning parameters.
`rake_params`	`Option<RakeParams>`	`None`	RAKE-specific tuning parameters.

Methods

default()

Signature:

pub fn default() -> KeywordConfig

Example:

let result = KeywordConfig::default();

Returns: KeywordConfig

KeywordExtractor

Post-processor that extracts keywords from document content.

This processor:

Runs in the Middle processing stage
Only processes when config.keywords is configured
Stores extracted keywords in metadata.additional["keywords"]
Uses the configured algorithm (YAKE or RAKE)

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

estimated_duration_ms()

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

LanguageConfidence

Since: v1.1

Structured per-language detection result: confidence, document share, and script — the information the ISO-code-only detected_languages list cannot convey (#261).

Populated by the language-detection processor alongside detected_languages, with one entry per language, in the same order as detected_languages.

Field	Type	Default	Description
`language`	`String`	—	ISO 639-3 language code, matching the corresponding entry in `detected_languages`.
`confidence`	`f64`	—	Confidence for this language, in `[0.0, 1.0]`. In single-language mode this is whatlang’s `Info::confidence()` for the whole document. In multi-language mode this is the average whatlang confidence across the document’s 200-character chunks that were classified as this language.
`proportion`	`f64`	—	Share of the document’s analyzed content classified as this language, in `[0.0, 1.0]`. In single-language mode this is always `1.0`. In multi-language mode this is the fraction of 200-character chunks classified as this language (chunks that did not meet `min_confidence` for any language are excluded from the count but still count toward the denominator).
`script`	`String`	—	Writing system whatlang detected for this language (e.g. `"Latin"`, `"Cyrillic"`).
`reliable`	`bool`	—	Whether this detection is considered reliable. In single-language mode this is whatlang’s own `Info::is_reliable()` (confidence above whatlang’s internal 0.9 threshold). In multi-language mode this is the chunk-averaged `confidence` above that same 0.9 threshold, since whatlang’s `is_reliable()` only applies to a single detection.

LanguageDetectionConfig

Language detection configuration.

Field	Type	Default	Description
`enabled`	`bool`	`true`	Enable language detection
`min_confidence`	`f64`	`0.8`	Minimum confidence threshold (0.0-1.0)
`detect_multiple`	`bool`	`false`	Detect multiple languages in the document

Methods

default()

Signature:

pub fn default() -> LanguageDetectionConfig

Example:

let result = LanguageDetectionConfig::default();

Returns: LanguageDetectionConfig

LanguageDetector

Post-processor that detects languages in document content.

This processor:

Runs in the Early processing stage
Only processes when config.language_detection is configured
Stores detected languages in result.detected_languages
Stores structured per-language confidence, proportion, script, and reliability in result.detected_language_confidences (#261)
Uses the whatlang library for detection

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

estimated_duration_ms()

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

LateInteractionConfig

Configuration for the late-interaction (ColBERT) pipeline.

Controls which model to use, batching, and download/cache behavior for the local ONNX ColBERT model.

Field	Type	Default	Description
`model`	`LateInteractionModelType`	`LateInteractionModelType::Preset`	The late-interaction model to use (defaults to the “gte-moderncolbert” preset).
`batch_size`	`usize`	`16`	Batch size for local ONNX inference. ColBERT emits a `[seq, dim]` multi-vector embedding per document, so memory scales with batch size — keep this modest.
`max_length`	`usize`	`512`	Maximum token sequence length for the tokenizer (documents).
`query_max_length`	`usize`	`32`	Fixed padded length for query augmentation. ColBERT queries are padded (with the mask token, kept attention-live) to exactly this many tokens rather than truncated/left as-is — this is the “query augmentation” trick from the ColBERT paper.
`show_download_progress`	`bool`	`false`	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at `info` level on the `xberg::model_download` target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by `LateInteractionModelType::Plugin`, which downloads no model.
`cache_dir`	`Option<PathBuf>`	`None`	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for the late-interaction ONNX model.
`max_embed_duration_secs`	`Option<u64>`	`60`	Maximum wall-clock duration (in seconds) for a single embed call when using `LateInteractionModelType::Plugin`. `None` disables the timeout.

Methods

default()

Signature:

pub fn default() -> LateInteractionConfig

Example:

let result = LateInteractionConfig::default();

Returns: LateInteractionConfig

LateInteractionEngine

ColBERT late-interaction (multi-vector) model with thread-safe inference.

Rust-only: an opaque ORT-backed handle with no faithful binding representation (mirrors reranking::engine::RerankerEngine). Bindings drive inference through the module-level functions, not this type.

LateInteractionMatch

A single document match returned by max_sim_rank, with its position in the input and MaxSim score.

Field	Type	Default	Description
`index`	`usize`	—	Position of this document in the original input slice.
`score`	`f32`	—	MaxSim relevance score. Higher means more relevant to the query.

LateInteractionPreset

Static metadata for a bundled ColBERT preset (WASM/Android-safe, no ORT).

Field	Type	Default	Description
`name`	`String`	—	Stable preset name referenced from config.
`model_repo`	`String`	—	HuggingFace repository hosting the ONNX model.
`model_file`	`String`	—	Path to the ONNX file within the repo.
`additional_files`	`Vec<String>`	—	Sibling files that must be downloaded alongside `model_file`.
`max_length`	`usize`	—	Maximum document token sequence length.
`query_max_length`	`usize`	—	Fixed padded query length (ColBERT query augmentation).
`dim`	`usize`	—	Per-token embedding dimensionality.
`description`	`String`	—	Human-readable description.

LatexExtractor

LaTeX document extractor

Methods

default()

Signature:

pub fn default() -> LatexExtractor

Example:

let result = LatexExtractor::default();

Returns: LatexExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

LayoutDetection

A single layout detection result.

Field	Type	Default	Description
`class_name`	`LayoutClass`	—	Detected layout class (e.g. `Table`, `Text`, `Title`).
`confidence`	`f32`	—	Detection confidence score in `[0.0, 1.0]`.
`bbox`	`BBox`	—	Bounding box in image pixel coordinates.

LayoutDetectionConfig

Layout detection configuration.

Controls layout detection behavior in the extraction pipeline. When set on ExtractionConfig, layout detection is enabled for PDF extraction.

Field	Type	Default	Description
`strategy`	`LayoutStrategy`	`LayoutStrategy::Always`	Which pages the layout model runs on. Defaults to `LayoutStrategy::Always`, the historical behavior: every page is rendered and inferred. `LayoutStrategy::Auto` pre-screens pages with cheap signals and skips the model where it cannot help.
`confidence_threshold`	`Option<f32>`	`None`	Confidence threshold override (None = use model default).
`apply_heuristics`	`bool`	`true`	Whether to apply postprocessing heuristics (default: true).
`table_model`	`TableModel`	`TableModel::Tatr`	Table structure recognition model. Controls which model is used for table cell detection within layout-detected table regions. Defaults to `TableModel::Tatr`.
`formula_model`	`Option<FormulaModel>`	`None`	Formula recognition model for layout-detected formula regions. `None` (the default) keeps the plain OCR text of the region. Setting a model converts each formula region crop to LaTeX. Requires the `formula-recognition` feature; without it the setting is ignored.
`table_overlap_preference`	`TableOverlapPreference`	`TableOverlapPreference::Content`	How to resolve overlapping native vs layout tables. When a native table and a layout (TATR/SLANeXT) table overlap on the same region, this controls which one is kept. Defaults to `TableOverlapPreference::Content` (historical behavior: keep the table with more content). Set to `TableOverlapPreference::Native` to favor source reading order (higher text F1) over the model’s cell reflow.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for ONNX models (layout detection + table structure). When set, controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for inference. Defaults to `None` (auto-select per platform).
`enable_chart_understanding`	`bool`	`false`	Route regions classified as charts to the chart-understanding OCR task. When `true`, layout regions detected as charts are sent to the VLM chart task (data-series/axis recovery) instead of being treated as generic image regions. Defaults to `false` — chart understanding is opt-in and has no effect on standard text/table extraction scores.

Methods

default()

Signature:

pub fn default() -> LayoutDetectionConfig

Example:

let result = LayoutDetectionConfig::default();

Returns: LayoutDetectionConfig

LayoutEngine

High-level layout detection engine.

Wraps model loading, inference, and postprocessing into a single reusable object. Models are downloaded and cached on first use.

LayoutEngineConfig

Full configuration for the layout engine.

Provides fine-grained control over model selection, thresholds, and postprocessing.

Field	Type	Default	Description
`backend`	`ModelBackend`	`ModelBackend::RtDetr`	Which model backend to use.
`confidence_threshold`	`Option<f32>`	`None`	Confidence threshold override (None = use model default).
`apply_heuristics`	`bool`	`true`	Whether to apply postprocessing heuristics.
`cache_dir`	`Option<PathBuf>`	`None`	Custom cache directory for model files (None = default).
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for ONNX inference.

Methods

default()

Signature:

pub fn default() -> LayoutEngineConfig

Example:

let result = LayoutEngineConfig::default();

Returns: LayoutEngineConfig

LayoutModel

Common interface for all layout detection model backends.

Methods

detect()

Run layout detection on an image using the default confidence threshold.

Signature:

pub fn detect(&mut self, img: &RgbImage) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect(&todo!())?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The rgb image

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_with_threshold()

Run layout detection with a custom confidence threshold.

Signature:

pub fn detect_with_threshold(&mut self, img: &RgbImage, threshold: f32) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect_with_threshold(&todo!(), 0.5)?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The rgb image
`threshold`	`f32`	Yes	The threshold

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_batch()

Run layout detection on a batch of images in a single model call.

Returns one Vec<LayoutDetection> per input image (same order). threshold overrides the model’s default confidence cutoff when Some.

The default implementation is a sequential fallback: models that support true batched inference (e.g. rtdetr::RtDetrModel) override this.

Signature:

pub fn detect_batch(&mut self, images: &[&RgbImage], threshold: Option<f32>) -> Result<Vec<Vec<LayoutDetection>>, LayoutError>

Example:

let result = instance.detect_batch(&vec![], Some(0.5))?;

Parameters:

Name	Type	Required	Description
`images`	`&\[&RgbImage\]`	Yes	The images
`threshold`	`Option<f32>`	No	The threshold

Returns: Vec<Vec<LayoutDetection>>

Errors: Returns Err(LayoutError).

name()

Human-readable model name.

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

LayoutModelManager

Manages layout model downloading, caching, and path resolution.

LayoutRegion

A detected layout region on a page.

When layout detection is enabled, each page may have layout regions identifying different content types (text, pictures, tables, etc.) with confidence scores and spatial positions.

Field	Type	Default	Description
`class_name`	`String`	—	Layout class name (e.g. “picture”, “table”, “text”, “section_header”).
`confidence`	`f64`	—	Confidence score from the layout detection model (0.0 to 1.0).
`bounding_box`	`BoundingBox`	—	Bounding box in document coordinate space.
`area_fraction`	`f64`	—	Fraction of the page area covered by this region (0.0 to 1.0).

LinkMetadata

Link element metadata.

Field	Type	Default	Description
`href`	`String`	—	The href URL value
`text`	`String`	—	Link text content (normalized)
`title`	`Option<String>`	`None`	Optional title attribute
`link_type`	`LinkType`	—	Link type classification
`rel`	`Vec<String>`	—	Rel attribute values
`attributes`	`Vec<KeyValueAttribute>`	—	Additional attributes as key-value pairs.

ListItemMetadata

Metadata about a detected list item.

Field	Type	Default	Description
`list_type`	`ListType`	—	Type of list (Bullet, Numbered, etc.)
`byte_start`	`usize`	—	Starting byte offset in the content string
`byte_end`	`usize`	—	Ending byte offset in the content string
`indent_level`	`u32`	—	List item indent level

LlmBudgetConfig

Since: v1.1

Budget enforcement configuration.

Mirrors liter-llm’s LlmBudgetConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not enforced at request time.

Field	Type	Default	Description
`global_limit`	`Option<f64>`	`Default::default()`	Global spend limit in USD.
`model_limits`	`Option<HashMap<String, f64>>`	`HashMap::new()`	Per-model spend limits in USD, keyed by model name.
`enforcement`	`Option<String>`	`Default::default()`	Enforcement mode: `"hard"` (reject over-budget requests) or `"soft"` (log only).

LlmCacheConfig

Since: v1.1

Response cache configuration.

Mirrors liter-llm’s LlmCacheConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not consulted at request time.

Field	Type	Default	Description
`max_entries`	`Option<usize>`	`Default::default()`	Maximum number of cached entries.
`ttl_seconds`	`Option<u64>`	`Default::default()`	Cache entry time-to-live, in seconds.
`backend`	`Option<String>`	`Default::default()`	Cache backend name (e.g. `"memory"`, or an `opendal` scheme).
`backend_config`	`Option<HashMap<String, String>>`	`HashMap::new()`	Backend-specific configuration key/value pairs.

LlmConfig

Configuration for an LLM provider/model via liter-llm.

Each feature (VLM OCR, VLM embeddings, structured extraction) carries its own LlmConfig, allowing different providers per feature.

Debug is implemented by hand so api_key, header values, and the AWS credentials in BedrockConfig are never printed.

Field	Type	Default	Description
`model`	`String`	—	Provider/model string using liter-llm routing format. Examples: `"openai/gpt-4o"`, `"anthropic/claude-sonnet-4-20250514"`, `"groq/llama-3.1-70b-versatile"`.
`api_key`	`Option<String>`	`Default::default()`	API key for the provider. When `None`, liter-llm falls back to the provider’s standard environment variable (e.g., `OPENAI_API_KEY`).
`base_url`	`Option<String>`	`Default::default()`	Custom base URL override for the provider endpoint.
`timeout_secs`	`Option<u64>`	`Default::default()`	Request timeout in seconds. When `None`, liter-llm’s built-in 60s default applies, except the VLM OCR path which uses a 300s default (a single page image transcription routinely exceeds 60s). Set explicitly to override.
`max_retries`	`Option<u32>`	`Default::default()`	Maximum retry attempts (default: 3).
`temperature`	`Option<f64>`	`Default::default()`	Sampling temperature for generation tasks.
`max_tokens`	`Option<u64>`	`Default::default()`	Maximum tokens to generate.
`top_p`	`Option<f64>`	`Default::default()`	Nucleus sampling parameter for generation tasks, applied to individual requests built from this config. Restricts sampling to the smallest set of tokens whose cumulative probability mass is at least this value; lower is more focused. Validated to `[0.0, 1.0]` by `LlmConfig::validate`. Mirrors liter-llm’s `ChatCompletionRequest::top_p`. A request-time parameter like `temperature`/`max_tokens` above, not a client-level setting.
`stop`	`Option<Vec<String>>`	`vec![]`	Stop sequence(s) that halt token generation, applied to individual requests built from this config. Mirrors liter-llm’s `ChatCompletionRequest::stop` (`types::common::StopSequence`), which liter-llm represents as either a single string or a list of strings via an untagged enum. Always expressed here as a list — even one stop sequence is `["..."]` — so the field has a single, FFI-friendly shape across every language binding instead of a single-or-list union type. Converted to liter-llm’s `StopSequence::Multiple` at each request-building call site; see `llm::client::to_stop_sequence`.
`seed`	`Option<i64>`	`Default::default()`	Random seed for reproducible outputs, applied to individual requests built from this config. Provider support varies — some silently ignore it. Mirrors liter-llm’s `ChatCompletionRequest::seed`.
`presence_penalty`	`Option<f64>`	`Default::default()`	Presence penalty for generation tasks, applied to individual requests built from this config. Positive values discourage the model from repeating topics already present in the conversation. Validated to `[-2.0, 2.0]` by `LlmConfig::validate`. Mirrors liter-llm’s `ChatCompletionRequest::presence_penalty`.
`frequency_penalty`	`Option<f64>`	`Default::default()`	Frequency penalty for generation tasks, applied to individual requests built from this config. Positive values discourage the model from repeating the same tokens verbatim. Validated to `[-2.0, 2.0]` by `LlmConfig::validate`. Mirrors liter-llm’s `ChatCompletionRequest::frequency_penalty`.
`reasoning_effort`	`Option<String>`	`Default::default()`	Reasoning effort level for extended-thinking models, applied to individual requests built from this config. Mirrors liter-llm’s `ChatCompletionRequest::reasoning_effort` (`types::chat::ReasoningEffort`). A request-time parameter like `temperature`/ `max_tokens` above, not a client-level setting — `into_client_builder` does not map it. Accepted as a plain string — one of `"low"`, `"medium"`, `"high"`, `"minimal"`, `"max"` (case-insensitive; liter-llm’s own `#[serde(rename_all = "lowercase")]` spelling) — rather than importing liter-llm’s enum, because this module compiles even when the `liter-llm` feature is disabled. See `llm::client::parse_reasoning_effort` for the conversion into `liter_llm::ReasoningEffort`.
`extra_body`	`Option<serde_json::Value>`	`Default::default()`	Provider-specific extra parameters merged into the request body (guardrails, safety settings, grounding config, etc.), applied to individual requests built from this config. Mirrors liter-llm’s `ChatCompletionRequest::extra_body`. A request-time parameter like `temperature`/`max_tokens` above, not a client-level setting.
`load_env`	`Option<bool>`	`Default::default()`	Whether liter-llm should load provider credentials from environment variables. Mirrors liter-llm’s `ClientConfigBuilder::load_env`. When `None`, liter-llm’s own default behavior applies.
`headers`	`Option<HashMap<String, String>>`	`HashMap::new()`	Extra HTTP headers sent with every request to the provider. Mirrors liter-llm’s `ClientConfigBuilder::header`, for gateways or providers that require custom auth/routing headers.
`providers`	`Option<Vec<LlmProviderConfig>>`	`vec![]`	Custom provider configurations, in addition to liter-llm’s built-in providers. Mirrors liter-llm’s `LlmConfig::providers`, for OpenAI-compatible gateways and self-hosted model servers that are not in the built-in provider catalog.
`cache`	`Option<LlmCacheConfig>`	`Default::default()`	Response cache configuration. Mirrors liter-llm’s `LlmConfig::cache`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused. Boxed for the same reason as `bedrock` (a4579589ac): `LlmConfig` is the payload of `EmbeddingModelType::Llm` and `RerankerModelType::Llm`, whose other variants are tens of bytes. Inlining this and the two sub-configs below pushed that variant to 480 bytes and tripped `clippy::large_enum_variant` on the `--features full` leg.
`budget`	`Option<LlmBudgetConfig>`	`Default::default()`	Budget enforcement configuration. Mirrors liter-llm’s `LlmConfig::budget`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`rate_limit`	`Option<LlmRateLimitConfig>`	`Default::default()`	Per-model rate limiting configuration. Mirrors liter-llm’s `LlmConfig::rate_limit`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`cost_tracking`	`Option<bool>`	`Default::default()`	Enable per-request cost tracking. Mirrors liter-llm’s `LlmConfig::cost_tracking`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`tracing`	`Option<bool>`	`Default::default()`	Enable OpenTelemetry-compatible tracing spans. Mirrors liter-llm’s `LlmConfig::tracing`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`cooldown_secs`	`Option<u64>`	`Default::default()`	Cooldown duration after transient errors, in seconds. Mirrors liter-llm’s `LlmConfig::cooldown_secs`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`health_check_secs`	`Option<u64>`	`Default::default()`	Background health check interval, in seconds. Mirrors liter-llm’s `LlmConfig::health_check_secs`. Only takes effect when liter-llm’s `tower` feature is compiled in; otherwise the value is accepted but unused.
`bedrock`	`Option<BedrockConfig>`	`Default::default()`	AWS Bedrock settings (region, cross-region routing, explicit credentials). Only consulted for `bedrock/`-prefixed models. When `None` — or when an individual field inside it is `None` — liter-llm falls back to the standard AWS environment variables and the default credential chain.
`credential_provider`	`Option<CredentialProviderConfig>`	`Default::default()`	Managed OAuth2/STS credential provider for auth modes liter-llm cannot express via a static `api_key` — Azure AD, Vertex AI OAuth2, Vertex AI Application Default Credentials, and AWS STS Web Identity (EKS IRSA) for Bedrock. Mirrors liter-llm’s `client::ClientConfigBuilder::credential_provider`, which takes an `Arc<dyn liter_llm::auth::CredentialProvider>` trait object — that cannot appear in a serde DTO. Every `CredentialProviderConfig` variant is plain data instead, so it round-trips through TOML/JSON/YAML and every language binding like the rest of `LlmConfig`. Managed credential providers are unavailable on `wasm32`, where liter-llm uses browser HTTP rather than its native authentication modules. `LlmConfig::validate` rejects a configured provider on that target instead of silently ignoring it. GitHub Copilot’s device-flow provider has no variant here: it takes no configuration at all (`liter_llm::auth::github_copilot::GithubCopilotCredentialProvider::new` accepts only an HTTP client) and drives an interactive terminal prompt, so it cannot be expressed as data. A Rust embedder who needs it — or any other fully custom `CredentialProvider` — can call `xberg::llm::client::create_client_with_credential_provider` directly with a `liter-llm` dependency of their own.
`max_concurrency`	`Option<usize>`	`Default::default()`	Maximum number of simultaneously in-flight requests to the LLM provider this config resolves to. This is a real, global bound on provider concurrency, not a per-extraction allowance: `xberg::llm::client::create_client` shares one process-wide client instance per distinct resolved config, so every concurrent extraction that resolves to the same config shares the one in-flight-request limit this value configures, instead of each minting its own (GH#1465). `None` means unlimited. PDF and image OCR batch sizing are not derived from this field, even when the configured OCR backend or `vlm_fallback` policy can reach a VLM — those call sites mix CPU-bound raster/OCR work with, at most, occasional remote requests, so they size their batches from the general thread budget (`max_threads`) unconditionally (GH#1465). Captioning is the one feature that additionally uses this value to bound its own per-extraction async request fan-out (issuing only VLM requests, with no CPU batching to protect), on top of the global provider-side limit described above; that per-extraction bound clamps a value below 1 up to 1. The global provider-side limit does not clamp: `Some(0)` reaches liter-llm as-is, and liter-llm rejects it when building the client (zero permitted in-flight requests is never useful), so it surfaces as a `create_client` error rather than a silent clamp to 1. This field is intentionally last to preserve positional constructor compatibility in generated language bindings.
`max_response_bytes`	`Option<usize>`	`Default::default()`	Maximum size, in bytes, of a single HTTP response body read from the LLM provider. Bounds response bodies on every non-streaming call (chat completions, embeddings, model listings, …) and the error body read on a failed request; a successful streaming response keeps its own existing frame bounds and is unaffected. `None` (the default) means unbounded, matching liter-llm’s own default. Mirrors liter-llm’s `client::ClientConfigBuilder::max_response_bytes`, which is native-only (`native-http`, non-`wasm32`) — see `llm::client::build_client_config`. `Some(0)` is rejected by `LlmConfig::validate` rather than reaching liter-llm, which would refuse it at client-build time with the same complaint.

Methods

validate()

Validate the request-time sampling parameters that have a documented range: top_p ([0.0, 1.0]), presence_penalty, and frequency_penalty (both [-2.0, 2.0], matching liter-llm’s/OpenAI’s semantics). An unset field is always valid — silence in config should never be rejected.

Called from llm::client::build_client_config before a liter-llm client is built from this config, alongside the existing validate_cache_backend check in that function.

Signature:

pub fn validate(&self) -> Result<(), Error>

Example:

instance.validate()?;

Returns: No return value.

Errors: Returns Err(Error).

LlmProviderConfig

Since: v1.1

A custom provider configuration entry, in addition to liter-llm’s built-in providers.

Mirrors liter-llm’s LlmProviderConfig.

Field	Type	Default	Description
`name`	`String`	—	Provider name, used to key model prefix matching.
`base_url`	`String`	—	Base URL for the provider’s OpenAI-compatible API.
`auth_header`	`Option<String>`	`Default::default()`	Header name used to carry the API key (defaults to `Authorization` when unset).
`model_prefixes`	`Vec<String>`	`vec![]`	Model name prefixes routed to this provider (e.g. `["my-provider/"]`).

LlmRateLimitConfig

Since: v1.1

Per-model rate limiting configuration.

Mirrors liter-llm’s LlmRateLimitConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not enforced at request time.

Field	Type	Default	Description
`rpm`	`Option<u32>`	`Default::default()`	Requests per minute limit.
`tpm`	`Option<u64>`	`Default::default()`	Tokens per minute limit.
`window_seconds`	`Option<u64>`	`Default::default()`	Rate limit window, in seconds.

LlmUsage

Token usage and cost data for a single LLM call made during extraction.

Populated when VLM OCR, structured extraction, or LLM-based embeddings are used. Multiple entries may be present when multiple LLM calls occur within one extraction (e.g. VLM OCR + structured extraction).

Field	Type	Default	Description
`model`	`String`	—	The LLM model identifier (e.g. “openai/gpt-4o”, “anthropic/claude-sonnet-4-20250514”).
`source`	`String`	—	The pipeline stage that triggered this LLM call (e.g. “vlm_ocr”, “structured_extraction”, “embeddings”).
`input_tokens`	`Option<u64>`	`Default::default()`	Number of input/prompt tokens consumed.
`output_tokens`	`Option<u64>`	`Default::default()`	Number of output/completion tokens generated.
`total_tokens`	`Option<u64>`	`Default::default()`	Total tokens (input + output).
`estimated_cost`	`Option<f64>`	`Default::default()`	Estimated cost in USD based on the provider’s published pricing.
`finish_reason`	`Option<String>`	`Default::default()`	Why the model stopped generating (e.g. “stop”, “length”, “content_filter”).

ManifestEntryResponse

Model manifest entry for cache management.

Field	Type	Default	Description
`relative_path`	`String`	—	Relative path within the cache directory
`sha256`	`String`	—	SHA256 checksum of the model file
`size_bytes`	`u64`	—	Expected file size in bytes
`source_url`	`String`	—	HuggingFace source URL for downloading

ManifestResponse

Model manifest response.

Field	Type	Default	Description
`xberg_version`	`String`	—	Xberg version
`total_size_bytes`	`u64`	—	Total size of all models in bytes
`model_count`	`usize`	—	Number of models in the manifest
`models`	`Vec<ManifestEntryResponse>`	—	Individual model entries

MapResult

The result of a map operation, containing discovered URLs.

Field	Type	Default	Description
`urls`	`Vec<SitemapUrl>`	`vec![]`	The list of discovered URLs.

MarkdownCodeBlock

A fenced code block extracted from Markdown.

Field	Type	Default	Description
`language`	`String`	—	Declared language identifier, or an empty string when absent.
`code`	`String`	—	Code block content.

MarkdownExtractor

Markdown extractor with metadata and table support.

Parses markdown documents with YAML frontmatter, extracting:

Metadata from YAML frontmatter
Plain text content
Tables as structured data
Document structure (headings, links, code blocks)
Images from data URIs

Methods

default()

Signature:

pub fn default() -> MarkdownExtractor

Example:

let result = MarkdownExtractor::default();

Returns: MarkdownExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

MarkdownLink

A link extracted from Markdown.

Field	Type	Default	Description
`text`	`String`	—	Visible link text.
`url`	`String`	—	Link destination.

MdxExtractor

MDX extractor with JSX stripping and Markdown processing.

Strips MDX-specific syntax (imports, exports, JSX component tags, inline expressions) and processes the remaining content as Markdown, extracting metadata from YAML frontmatter and tables.

Methods

default()

Signature:

pub fn default() -> MdxExtractor

Example:

let result = MdxExtractor::default();

Returns: MdxExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

MergedChunk

A merged chunk produced by merge_segments.

Field	Type	Default	Description
`text`	`String`	—	Merged text content (may include overlap from the previous chunk).
`byte_start`	`usize`	—	Byte offset of the first character in the original document.
`byte_end`	`usize`	—	Byte offset one past the last character in the original document.

MetaSchema

Compiled meta-schema validator over preset.schema.json.

Methods

compile()

Compile the given JSON text as a Draft 2020-12 meta-schema.

Signature:

pub fn compile(meta_schema_json: &str) -> Result<MetaSchema, LoadError>

Example:

let result = MetaSchema::compile("value")?;

Parameters:

Name	Type	Required	Description
`meta_schema_json`	`&str`	Yes	The meta schema json

Returns: MetaSchema

Errors: Returns Err(LoadError).

parse_preset()

Validate raw against the meta-schema and deserialize into a Preset, stamping the fingerprint over the canonical file bytes.

Signature:

pub fn parse_preset(&self, path: &str, raw: &[u8]) -> Result<Preset, LoadError>

Example:

let result = instance.parse_preset("value", b"data")?;

Parameters:

Name	Type	Required	Description
`path`	`&str`	Yes	Path to the file
`raw`	`&\[u8\]`	Yes	The raw

Returns: Preset

Errors: Returns Err(LoadError).

Metadata

Extraction result metadata.

Contains common fields applicable to all formats, format-specific metadata via a discriminated union, and additional custom fields from postprocessors.

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Document title
`subject`	`Option<String>`	`Default::default()`	Document subject or description
`authors`	`Option<Vec<String>>`	`vec![]`	Primary author(s) - always Vec for consistency
`keywords`	`Option<Vec<String>>`	`vec![]`	Keywords/tags - always Vec for consistency
`language`	`Option<String>`	`Default::default()`	Primary language (ISO 639 code)
`created_at`	`Option<String>`	`Default::default()`	Creation timestamp (ISO 8601 format)
`modified_at`	`Option<String>`	`Default::default()`	Last modification timestamp (ISO 8601 format)
`created_by`	`Option<String>`	`Default::default()`	User who created the document
`modified_by`	`Option<String>`	`Default::default()`	User who last modified the document
`pages`	`Option<PageStructure>`	`Default::default()`	Page/slide/sheet structure with boundaries
`format`	`Option<FormatMetadata>`	`Default::default()`	Format-specific metadata (discriminated union) Contains detailed metadata specific to the document format. Serialized as a nested `"format"` object with a `format_type` discriminator field.
`image_preprocessing`	`Option<ImagePreprocessingMetadata>`	`Default::default()`	Image preprocessing metadata (when OCR preprocessing was applied)
`json_schema`	`Option<serde_json::Value>`	`Default::default()`	JSON schema (for structured data extraction)
`error`	`Option<ErrorMetadata>`	`Default::default()`	Error metadata (for batch operations)
`extraction_duration_ms`	`Option<u64>`	`Default::default()`	Extraction duration in milliseconds (for benchmarking). This field is populated by batch extraction to provide per-file timing information. It’s `None` for single-file extraction (which uses external timing).
`category`	`Option<String>`	`Default::default()`	Document category (from frontmatter or classification).
`tags`	`Option<Vec<String>>`	`vec![]`	Document tags (from frontmatter).
`document_version`	`Option<String>`	`Default::default()`	Document version string (from frontmatter).
`abstract_text`	`Option<String>`	`Default::default()`	Abstract or summary text (from frontmatter).
`output_format`	`Option<String>`	`Default::default()`	Output format identifier (e.g., “markdown”, “html”, “text”). Set by the output format pipeline stage when format conversion is applied. Previously stored in `metadata.additional["output_format"]`.
`ocr_used`	`bool`	—	Whether OCR was used during extraction. Set to `true` whenever the extraction pipeline ran an OCR backend (Tesseract, PaddleOCR, VLM, etc.) and used that output as the primary or fallback text. `false` means native text extraction was used exclusively.
`additional`	`HashMap<String, serde_json::Value>`	`HashMap::new()`	Additional custom fields from postprocessors. Serialized as a nested `"additional"` object (not flattened at root level). Uses `Cow<'static, str>` keys so static string keys avoid allocation.

Methods

is_empty()

Returns true when no metadata fields, format-specific metadata, or additional postprocessor fields are populated.

Signature:

pub fn is_empty(&self) -> bool

Example:

let result = instance.is_empty();

Returns: bool

MetricsLayer

A tower::Layer that records service-level extraction metrics.

ModelCacheStats

Statistics about the PaddleOCR model cache.

Field	Type	Default	Description
`total_size_bytes`	`u64`	—	Total size of cached models in bytes.
`model_count`	`usize`	—	Number of models currently cached.
`cache_dir`	`PathBuf`	—	Path to the cache directory.

ModelManager

Manages PaddleOCR model downloading, caching, and path resolution.

The model manager ensures that PaddleOCR models are available locally, organized by model type. Shared models (det, cls) are downloaded once, while recognition models are downloaded per-script-family on demand.

Methods

default()

Signature:

pub fn default() -> ModelManager

Example:

let result = ModelManager::default();

Returns: ModelManager

ModelManifestEntry

A single downloadable model entry.

Field	Type	Default	Description
`relative_path`	`String`	—	Stable logical path used by manifest consumers; the runtime artifact remains in Hugging Face’s snapshot layout rather than at this path.
`sha256`	`String`	—	SHA256 checksum of the model file.
`size_bytes`	`u64`	—	Expected file size in bytes.
`source_url`	`String`	—	HuggingFace source URL for downloading.

ModelParagraph

A single paragraph; may or may not carry a text payload.

Field	Type	Default	Description
`text`	`Option<ParaText>`	`Default::default()`	The decoded paragraph text, or `None` if this paragraph has no text record.
`outline_level`	`u8`	—	Outline level from the ParaShape record (0 = body text, 1–7 = headings).
`char_shape_runs`	`Vec<(u32, u16)>`	`vec![]`	Mappings from character position to char_shape index.
`equations`	`Vec<String>`	`vec![]`	The LaTeX of every equation this paragraph holds, in document order. The parser knows each equation exactly, because it substitutes the text itself. Reading it here is exact, where searching the finished text for `$` would take a price for an equation and an equation for a price.

ModelPaths

Combined paths to all models needed for OCR (backward compatibility).

Field	Type	Default	Description
`det_model`	`PathBuf`	—	Exact path to the detection ONNX model in the Hugging Face snapshot.
`cls_model`	`PathBuf`	—	Exact path to the classification ONNX model in the Hugging Face snapshot.
`rec_model`	`PathBuf`	—	Exact path to the recognition ONNX model in the Hugging Face snapshot.
`dict_file`	`PathBuf`	—	Path to the character dictionary file.

MultiVectorEmbedding

A ColBERT multi-vector embedding: one row per attention-live token.

data is a flat, row-major buffer of length num_tokens * dim — row i (the embedding for token i) occupies data[i*dim .. (i+1)*dim]. Flat storage keeps the type FFI-friendly across binding boundaries; use MultiVectorEmbedding::rows internally to iterate per-token slices.

Field	Type	Default	Description
`num_tokens`	`u32`	—	Number of attention-live token rows (padding rows are dropped, not zeroed — see `engine::normalize_tokens`).
`dim`	`u32`	—	Dimensionality of each per-token vector.
`data`	`Vec<f32>`	—	Flat row-major buffer, length `num_tokens * dim`.

Methods

is_well_formed()

Returns true if data holds exactly num_tokens * dim values — i.e. the flat buffer matches the declared shape.

All fields are pub and the type is Deserialize, so a value coming from an untrusted source (FFI caller, JSON, a store row) may be malformed. max_sim_score guards on this so a length-mismatched buffer scores 0.0 rather than silently mis-scoring (a shorter data would make rows’ chunks_exact drop a trailing partial chunk). Uses checked_mul so an overflowing num_tokens * dim is reported as malformed instead of wrapping.

Signature:

pub fn is_well_formed(&self) -> bool

Example:

let result = instance.is_well_formed();

Returns: bool

MultidocInput

Input signals for multi-document boundary detection.

Field	Type	Default	Description
`page_count`	`u32`	—	Total number of pages in the PDF.
`pages`	`Vec<PageSignals>`	—	Per-page signals extracted from the PDF.

MultidocThresholds

Thresholds for multi-document boundary detection.

All fields are public; callers override any subset via struct-update syntax.

Field	Type	Default	Description
`density_shift_threshold`	`f32`	`0.3`	Text density difference threshold for `DensityShift` detection. Default: 0.3.
`bigram_overlap_min`	`f32`	`0.1`	Minimum bigram-overlap ratio below which a density shift is promoted to a `DensityShift` boundary. Default: 0.1 (10 % overlap).

Methods

default()

Signature:

pub fn default() -> MultidocThresholds

Example:

let result = MultidocThresholds::default();

Returns: MultidocThresholds

NerConfig

Since: v1.0

Configuration for the NER post-processor.

Field	Type	Default	Description
`backend`	`NerBackendKind`	`NerBackendKind::Onnx`	Backend that runs the entity detection.
`categories`	`Vec<EntityCategory>`	`vec![]`	Entity categories to detect. Defaults to a sensible PERSON/ORG/LOCATION/EMAIL set when empty.
`model`	`Option<String>`	`Default::default()`	Override the default model — only used by `NerBackendKind::Onnx`. `None` lets the backend pick its pinned default xberg GLiNER model alias.
`llm`	`Option<LlmConfig>`	`Default::default()`	Optional LLM configuration — only used by `NerBackendKind::Llm`. Token usage for LLM backends is recorded in `ExtractedDocument::llm_usage`.
`custom_labels`	`Vec<String>`	`vec![]`	Arbitrary user-supplied entity labels for zero-shot detection. `xberg-gliner` natively supports zero-shot inference over caller-supplied labels. The LLM backend also honours these labels by including them in the structured-output schema. Custom labels surface as `EntityCategory::Custom` in the resulting `Entity` stream. Use this when you need domain-specific entity types (e.g. `"Treatment"`, `"Product"`, `"Vessel"`) without forking GLiNER’s taxonomy.

NerProcessor

NER post-processor.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

NgramRange

Inclusive word-count range used to form keyword candidates.

Field	Type	Default	Description
`min`	`usize`	`1`	Minimum number of words in a candidate.
`max`	`usize`	`3`	Maximum number of words in a candidate.

Methods

default()

Signature:

pub fn default() -> NgramRange

Example:

let result = NgramRange::default();

Returns: NgramRange

NormalizeResult

Result of image normalization

Field	Type	Default	Description
`rgb_data`	`Vec<u8>`	—	Processed RGB image data (height width 3 bytes)
`dimensions`	`(usize, usize)`	—	Image dimensions (width, height)
`metadata`	`ImagePreprocessingMetadata`	—	Preprocessing metadata

NumbersExtractor

Apple Numbers spreadsheet extractor.

Supports .numbers files (modern iWork format, 2013+).

Extracts table names and cell string values from the IWA container: ZIP → Snappy → typed protobuf table storage. Output preserves the source row and column structure.

Methods

default()

Signature:

pub fn default() -> NumbersExtractor

Example:

let result = NumbersExtractor::default();

Returns: NumbersExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

OcrBackend

Trait for OCR backend plugins.

Implement this trait to add custom OCR capabilities. OCR backends can be:

Native Rust implementations (like Tesseract)
FFI bridges to external libraries (like PaddleOCR)
Cloud-based OCR services (Google Vision, AWS Textract, etc.)

Thread Safety

OCR backends must be thread-safe (Send + Sync) to support concurrent processing.

Methods

process_image()

Process an image and extract text via OCR.

Returns:

An ExtractedDocument containing the extracted text and metadata.

Errors:

XbergError::Ocr - OCR processing failed
XbergError::Validation - Invalid image format or configuration
XbergError::Io - I/O errors (these always bubble up)

Reading `backend_options`

Backends that support runtime tuning can read config.backend_options and deserialize only the keys they care about. Unknown keys are silently ignored, so multiple backends can coexist in a pipeline without key conflicts.

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	Raw image data (JPEG, PNG, TIFF, etc.)
`config`	`&OcrConfig`	Yes	OCR configuration (language, PSM mode, etc.)

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Process a file and extract text via OCR.

Default implementation reads the file and calls process_image. Override for custom file handling or optimizations.

Errors:

Same as process_image, plus file I/O errors.

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the image file
`config`	`&OcrConfig`	Yes	OCR configuration

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Check if this backend supports a given language code.

Returns:

true if the language is supported, false otherwise.

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

fn supports_language(&self, lang: &str) -> bool {
    self.languages.contains(&lang.to_string())
}

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	ISO 639-2/3 language code (e.g., “eng”, “deu”, “fra”)

Returns: bool

backend_type()

Get the backend type identifier.

Returns:

The backend type enum value.

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

fn backend_type(&self) -> OcrBackendType {
    OcrBackendType::Tesseract
}

Returns: OcrBackendType

supported_languages()

Optional: Get a list of all supported languages.

Defaults to empty list. Override to provide comprehensive language support info.

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

supported_languages_for()

Optional: languages usable under config, defaulting to ignoring it. GH#1857.

Signature:

pub fn supported_languages_for(&self, config: &OcrConfig) -> Vec<String>

Example:

let result = instance.supported_languages_for(&todo!());

Parameters:

Name	Type	Required	Description
`config`	`&OcrConfig`	Yes	The configuration options

Returns: Vec<String>

supports_language_for()

Optional: whether language is supported under config. GH#1857.

Signature:

pub fn supports_language_for(&self, config: &OcrConfig, language: &str) -> bool

Example:

let result = instance.supports_language_for(&todo!(), "value");

Parameters:

Name	Type	Required	Description
`config`	`&OcrConfig`	Yes	The configuration options
`language`	`&str`	Yes	The language

Returns: bool

supports_table_detection()

Optional: Check if the backend supports table detection.

Defaults to false. Override if your backend can detect and extract tables.

Signature:

pub fn supports_table_detection(&self) -> bool

Example:

let result = instance.supports_table_detection();

Returns: bool

supports_document_processing()

Check if the backend supports direct document-level processing (e.g. for PDFs).

Defaults to false. Override if the backend has optimized document processing. PDF extraction uses this optimized path only when both effective page margins are zero; nonzero margins require per-page image processing so geometry can be filtered correctly.

Signature:

pub fn supports_document_processing(&self) -> bool

Example:

let result = instance.supports_document_processing();

Returns: bool

emits_structured_markdown()

Declare that this backend emits structured markdown directly (tables, headings, lists) and downstream layout reconstruction should be skipped.

Defaults to false — classical OCR backends (Tesseract, PaddleOCR classical) return plain text per detected region. End-to-end VLM backends (PaddleOCR-VL, GOT-OCR 2.0) emit markdown in one forward pass and should override this to true.

Signature:

pub fn emits_structured_markdown(&self) -> bool

Example:

let result = instance.emits_structured_markdown();

Returns: bool

confidence_semantics()

Declare how this backend’s reported page-level confidence must be interpreted.

Defaults to ConfidenceSemantics::Uncalibrated. This default is deliberately the least trusting option, not ConfidenceSemantics::Legibility: a new backend that reports some confidence number is not thereby safe to gate on, and defaulting to Legibility would let the next backend silently inherit a threshold calibrated for a different backend’s scale — exactly the failure this type exists to prevent (see the type’s doc comment). Override this only after validating that the reported number tracks legibility on a known scale.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

page_orientation_handling()

Declare how this backend copes with a page raster whose text is not upright.

Defaults to PageOrientationHandling::RequiresUpright. This default is deliberately the least capable option, not PageOrientationHandling::SelfCorrecting: a new backend must not silently inherit Tesseract’s ability to reconstruct reading order on a rotated raster and then quietly emit garbage the first time it faces one (see the type’s doc comment for the measured A/B behind this). Override this only after validating the backend’s actual behaviour on a rotated page.

Signature:

pub fn page_orientation_handling(&self) -> PageOrientationHandling

Example:

let result = instance.page_orientation_handling();

Returns: PageOrientationHandling

process_document()

Process a document file directly via OCR.

Only called if supports_document_processing returns true.

Signature:

pub async fn process_document(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_document("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	The path
`config`	`&OcrConfig`	Yes	The ocr config

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

OcrBackendCapabilities

A registered OCR backend’s declared name and language capabilities.

Returned by list_ocr_backend_capabilities. See that function’s documentation for the determinism guarantees and the important caveat about what an empty supported_languages means.

Field	Type	Default	Description
`name`	`String`	—	The backend’s registered name, as returned by `list_ocr_backends`.
`supported_languages`	`Vec<String>`	—	The languages this backend declares support for, via `OcrBackend::supported_languages`. An empty list means the backend does not enumerate its languages — it is not a statement that the backend supports no languages. `OcrBackend::supported_languages` is a defaulted trait method that returns `vec![]`, and not every backend overrides it: the VLM backend (`llm::vlm_ocr::VlmOcrBackend`) accepts every language via `supports_language` while inheriting the empty default here. Use `ocr_backend_supports_language` to decide whether one specific language is usable — never infer “unsupported” from an empty list. The order of this list is preserved exactly as the backend reported it and is not re-sorted. Tesseract’s order comes from enumerating installed tessdata files; PaddleOCR’s comes from its own `SUPPORTED_LANGUAGES` constant. Re-sorting would disagree with the precedence each backend’s own `supports_language` implementation uses internally. This is the no-override answer (`list_ocr_backend_capabilities`). Use `list_ocr_backend_capabilities_for` when the caller sets `OcrConfig.tessdata_path`: for Tesseract the language list is a property of the resolved tessdata directory, and the no-override chain can name a different directory than the one a job using that config will load from. See GH#1857.

OcrBackendRegistry

Registry for OCR backend plugins.

Manages OCR backends with backend type and language-based selection.

Thread Safety

The registry is thread-safe and can be accessed concurrently from multiple threads.

Methods

default()

Signature:

pub fn default() -> OcrBackendRegistry

Example:

let result = OcrBackendRegistry::default();

Returns: OcrBackendRegistry

OcrCache

File-backed msgpack cache for OCR extraction results, keyed by image + config hash.

OcrCacheStats

Statistics about the current state of the OCR result cache.

Field	Type	Default	Description
`total_files`	`usize`	—	Number of msgpack cache files on disk.
`total_size_mb`	`f64`	—	Total size of all cache files in megabytes.

OcrConfidence

Confidence scores for an OCR element.

Separates detection confidence (how confident that text exists at this location) from recognition confidence (how confident about the actual text content).

Field	Type	Default	Description
`detection`	`Option<f64>`	`Default::default()`	Detection confidence: how confident the OCR engine is that text exists here. PaddleOCR provides this as `box_score`, Tesseract doesn’t have a direct equivalent. Range: 0.0 to 1.0 (or None if not available).
`recognition`	`f64`	—	Recognition confidence: how confident about the text content. Range: 0.0 to 1.0.

OcrConfig

OCR configuration.

Field	Type	Default	Description
`enabled`	`bool`	`true`	Whether OCR is enabled. Setting `enabled: false` is a shorthand for `disable_ocr: true` on the parent `ExtractionConfig`. Images return metadata only; PDFs use native text extraction without OCR fallback. Defaults to `true`. When `false`, all other OCR settings are ignored.
`backend`	`String`	`"tesseract"`	OCR backend: tesseract, paddleocr, paddle-ocr, sceptre, or vlm. Sceptre uses ONNX Runtime on desktop/server and tract on supported mobile builds. Browser WebAssembly uses the separate byte-fed Sceptre worker API.
`language`	`Vec<String>`	`["eng"]`	Language code(s) for OCR recognition. Defaults to `["eng"]`. For Tesseract, languages are joined with “+”. A list is the canonical form and the only form accepted by the binding object APIs (Python, Node, PHP, WASM, etc.): `["eng", "deu"]`. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”). The four candle-based backends also use this list to decide which scripts are plausible in their output, dropping a line written in an unconfigured script.
`tesseract_config`	`Option<TesseractConfig>`	`None`	Tesseract-specific configuration (optional)
`output_format`	`Option<OutputFormat>`	`None`	Output format for OCR results (optional, for format conversion)
`paddle_ocr_config`	`Option<serde_json::Value>`	`None`	Legacy PaddleOCR configuration JSON. Deprecated since 1.3.2 and planned for removal in 2.0. Use `Self::paddle_ocr_settings` for typed access.
`paddle_ocr_settings`	`Option<PaddleOcrConfig>`	`None`	Typed PaddleOCR-specific configuration (optional). Any `PaddleOcrConfig` field can be set here; unset fields keep their defaults. In a config file, a JSON config, or a REST or MCP request body, an unknown key is a configuration error. A typed object built in a binding has only the fields it declares, so it ignores any other property, as every other typed config does. The most common overrides are `model_version` (`"pp-ocrv6"` default / `"pp-ocrv5"`) and `model_tier`. In TOML: `[ocr.paddle_ocr_settings] model_version = "pp-ocrv5" model_tier = "server"` The `XBERG_OCR_MODEL_VERSION` / `XBERG_OCR_MODEL_TIER` environment variables set the same two keys for env-configured servers (issue #1279).
`backend_options`	`Option<serde_json::Value>`	`None`	Arbitrary per-call options passed through to the backend unchanged. Custom OCR backends and built-in backends that support runtime tuning can read this value and deserialize the keys they care about. Keys unknown to the backend are silently ignored. This is the recommended extension point for per-call parameters that are not covered by the typed fields above (e.g. mode switching, preprocessing flags, inference batch size). Scope: when `pipeline` is `None`, this value is propagated to the primary stage of the auto-constructed pipeline. When `pipeline` is explicitly set, this field has no effect — the caller must set `OcrPipelineStage.backend_options` directly on the relevant stage(s) instead. Example: `{ "mode": "fast", "enable_layout": true, "timeout_ms": 5000 }`
`element_config`	`Option<OcrElementConfig>`	`None`	OCR element extraction configuration
`quality_thresholds`	`Option<OcrQualityThresholds>`	`None`	Quality thresholds for the native-text-to-OCR fallback decision. When None, uses compiled defaults (matching previous hardcoded behavior).
`pipeline`	`Option<OcrPipelineConfig>`	`None`	Multi-backend OCR pipeline configuration. When set, enables weighted fallback across multiple OCR backends based on output quality. When None, uses the single `backend` field (same as today).
`auto_rotate`	`bool`	`false`	Enable automatic page rotation based on orientation detection. When enabled, page orientation (0/90/180/270 degrees) is detected with an ONNX PP-LCNet document-orientation classifier — NOT Tesseract’s own `DetectOrientationScript()`/OSD, which this library does not call. If the page is rotated with high confidence, the image is corrected before recognition. Applies to every OCR backend; the tesseract-only `TesseractConfig::preprocessing` field of the same name is OR’d in on top of this one and affects tesseract alone. Independent of a page’s PDF `/Rotate` entry, which is handled separately on every OCR route regardless of this setting.
`vlm_fallback`	`VlmFallbackPolicy`	`VlmFallbackPolicy::Disabled`	Ergonomic VLM fallback policy. When set to anything other than `VlmFallbackPolicy::Disabled` and `OcrConfig::pipeline` is `None`, a multi-stage pipeline is synthesised automatically: - `VlmFallbackPolicy::OnLowQuality` → `[classical_stage, vlm_stage]` with the `quality_threshold` mapped onto `OcrQualityThresholds::pipeline_min_quality`. - `VlmFallbackPolicy::Always` → `[vlm_stage]` only. Requires `OcrConfig::vlm_config` to be `Some` when not `Disabled`. When `OcrConfig::pipeline` is explicitly set, this field is ignored.
`vlm_config`	`Option<LlmConfig>`	`None`	VLM (Vision Language Model) OCR configuration. Required when `backend` is `"vlm"` or when `vlm_fallback` is not `VlmFallbackPolicy::Disabled`. Uses liter-llm to send page images to a vision model for text extraction.
`vlm_prompt`	`Option<String>`	`None`	Custom Jinja2 prompt template for VLM OCR. When `None`, uses the default template. Available variables: - `{{ language }}` — The document language code (e.g., “eng”, “deu”).
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for ONNX Runtime models (e.g. PaddleOCR, layout detection). Not user-configurable via config files — injected at runtime from `ExtractionConfig::acceleration` before each `process_image` call.
`security_limits`	`Option<SecurityLimits>`	`None`	Security limits applied when decoding raw image bytes for OCR (GH#1554). Not user-configurable via config files — injected at runtime from `ExtractionConfig::security_limits` before each `process_image` call, the same pattern `Self::acceleration` uses. `ExtractionConfig::security_limits` remains the source of truth and overwrites this field whenever it carries a value; a value set here directly is honoured only when `ExtractionConfig` carries none, so the two cannot drift while a directly-set limit is no longer silently discarded (GH#1651). A backend consulting a `backend_options` override for this call may still let that override win, but in the absence of one this field is what backends should fall back to instead of `SecurityLimits::default()`. `None` means “use `SecurityLimits::default()`”, never “disable the check”.
`tessdata_bytes`	`Option<HashMap<String, Vec<u8>>>`	`None`	Caller-supplied Tesseract `traineddata` bytes per language code. Primary use case is the WASM build, which has no filesystem and cannot download tessdata at runtime. Native builds typically rely on `TessdataManager` and ignore this field. When present, the WASM Tesseract backend prefers these bytes over its compile-time-bundled English data. Skipped by serde to keep config files small — supply via the typed API at runtime.
`tessdata_path`	`Option<PathBuf>`	`None`	Runtime override for tessdata directory path. When set, uses this path as the highest-priority tessdata location, bypassing environment variables and cache directories. Useful for embedding pre-installed tessdata in applications. When `None`, uses the standard resolution chain: TESSDATA_PREFIX env, cache dir, system paths.
`numeric_repair`	`bool`	`false`	Repair OCR tokens that are clearly numeric but mis-punctuated: a dropped thousands separator, a decimal point misread for a grouping comma, or one number split into two tokens at a rendering gap (GH#1789). Defaults to `false`. Unlike the always-on list-marker repair (`crate::extractors::pdf::ocr::scoring::repair_ocr_list_markers`), this repair has no table-column context available at the point OCR text comes back as a flat string, so it cannot tell `"1.234,56"` (European) from `"1,234.56"` (US) apart on its own – it always assumes the US/UK convention (comma groups, period decimals). The rules read only the punctuation and digit counts, never whether a token is an amount, so each of these is rewritten too (GH#1836). Enable it only for documents whose numbers are amounts, and which use that convention: - A bare 4-9 digit integer becomes grouped, whatever it denotes: `"Year 2019"` -> `"Year 2,019"`, and likewise a ZIP code, a page number or a part number. Only a literal `"FY "` prefix is exempt. - A US-convention decimal with exactly three fraction digits and at most three integer digits becomes an amount: `"0.125"` -> `"0,125"`. Four fraction digits (`"0.7906"`) or a trailing `%` are left alone. - A lone digit followed by a single space and an already-grouped number is joined, which is indistinguishable from two adjacent table cells: `"5 12,000"` -> `"512,000"`. The repair is skipped entirely when `tesseract_config.output_format` is `"hocr"` or `"tsv"`, because it would re-punctuate the bare coordinate integers in that markup.

Methods

default()

Signature:

pub fn default() -> OcrConfig

Example:

let result = OcrConfig::default();

Returns: OcrConfig

OcrElement

A unified OCR element representing detected text with full metadata.

This is the primary type for structured OCR output, preserving all information from both Tesseract and PaddleOCR backends.

Field	Type	Default	Description
`text`	`String`	—	The recognized text content.
`geometry`	`OcrBoundingGeometry`	`OcrBoundingGeometry::Rectangle`	Bounding geometry (rectangle or quadrilateral).
`confidence`	`OcrConfidence`	—	Confidence scores for detection and recognition.
`level`	`OcrElementLevel`	`OcrElementLevel::Line`	Hierarchical level (word, line, block, page).
`rotation`	`Option<OcrRotation>`	`Default::default()`	Rotation information (if detected).
`page_number`	`u32`	`1`	Page number (1-indexed).
`parent_id`	`Option<String>`	`Default::default()`	Parent element ID for hierarchical relationships. When hierarchy output is enabled, this resolves to another emitted element’s `backend_metadata["element_id"]` value.
`backend_metadata`	`HashMap<String, serde_json::Value>`	`HashMap::new()`	Backend-specific metadata that doesn’t fit the unified schema.

OcrElementConfig

Configuration for OCR element extraction.

Controls how OCR elements are extracted and filtered.

Field	Type	Default	Description
`include_elements`	`bool`	—	Whether to include OCR elements in the extraction result. When true, the `ocr_elements` field in `ExtractedDocument` will be populated.
`min_level`	`OcrElementLevel`	`OcrElementLevel::Line`	Minimum hierarchical level to include. Elements below this level (e.g., words when min_level is Line) will be excluded.
`min_confidence`	`f64`	—	Minimum recognition confidence threshold (0.0-1.0). Elements with confidence below this threshold will be filtered out.
`build_hierarchy`	`bool`	—	Whether to build hierarchical relationships between elements. When true, emitted elements receive an `element_id` metadata value and `parent_id` references are populated only when a spatially containing parent is also emitted.

OcrExtractionResult

OCR extraction result.

Result of performing OCR on an image or scanned document, including recognized text and detected tables.

Field	Type	Default	Description
`content`	`String`	—	Recognized text content
`mime_type`	`String`	—	Original MIME type of the processed image
`metadata`	`HashMap<String, serde_json::Value>`	`HashMap::new()`	OCR processing metadata (confidence scores, language, etc.)
`tables`	`Vec<OcrTable>`	`vec![]`	Tables detected and extracted via OCR
`ocr_elements`	`Option<Vec<OcrElement>>`	`vec![]`	Structured OCR elements with bounding boxes and confidence scores. Available when TSV output is requested or table detection is enabled.

OcrMetadata

OCR processing metadata.

Captures information about OCR processing configuration and results.

Field	Type	Default	Description
`language`	`String`	—	OCR language code(s) used
`psm`	`i32`	—	Tesseract Page Segmentation Mode (PSM)
`output_format`	`String`	—	Output format (e.g., “text”, “hocr”)
`table_count`	`u32`	—	Number of tables detected
`table_rows`	`Option<u32>`	`Default::default()`	Number of rows in the detected table (if a single table was found).
`table_cols`	`Option<u32>`	`Default::default()`	Number of columns in the detected table (if a single table was found).

OcrPageCoordinateFrame

One OCR page’s authoritative processed-raster coordinate frame (GH#1645).

Public OcrElement geometry stays in the OCR backend’s own raster pixel space even after the rest of a page’s document is normalized into PDF page points, and Metadata::additional’s document-wide ocr_processed_image_width/height pair cannot describe differently sized, preprocessed, or rotated pages. This record is the per-page authority a multi-page consumer joins to an OcrElement through the element’s own page_number.

Crate-private and carried on InternalDocument::ocr_coordinate_frame with #[serde(skip)] so it never crosses the plugin-bridge JSON wire format or gets an alef binding DTO; it is folded into Metadata::additional under ocr_metadata_keys::OCR_PAGE_COORDINATE_FRAMES_METADATA_KEY once a page’s document is final (extractors::pdf::mod).

Field	Type	Default	Description
`page_number`	`u32`	—	1-based page this frame describes.
`width`	`u32`	—	Processed raster width in pixels.
`height`	`u32`	—	Processed raster height in pixels.
`unit`	`String`	—	Always `"pixel"`.
`origin`	`String`	—	Always `"top_left"`.

OcrPageFailure

Since: v1.3

A page whose OCR failed while extraction of the document continued.

The warning in ExtractedDocument::processing_warnings stays available. One warning can cover several pages: when an automatic OCR run of selected pages fails as a whole, it adds one warning and one record for each selected page. A failure of the OCR fallback for the whole document adds a warning and no record. Repeated failures on one page make one record.

Field	Type	Default	Description
`page`	`u32`	—	One-based page number in the source document.
`error`	`String`	—	OCR backend error text.
`recovered`	`bool`	—	Whether content for this page was retained or recovered through another route.

OcrPipelineConfig

Multi-backend OCR pipeline with quality-based fallback.

Backends are tried in priority order (highest first). After each backend produces output, quality is evaluated. If it meets quality_thresholds.pipeline_min_quality, the result is accepted. Otherwise the next backend is tried; if none clears the threshold, an internal selection policy derived from the OcrConfig decides which stage’s result is returned as the best effort (vlm_fallback pipelines prefer their last non-empty stage; explicit and classical pipelines stay score-based).

Field	Type	Default	Description
`stages`	`Vec<OcrPipelineStage>`	—	Ordered list of backends to try. Sorted by priority (descending) at runtime.
`quality_thresholds`	`OcrQualityThresholds`	`/* serde(default) */`	Quality thresholds for deciding whether to accept a result or try the next backend.

OcrPipelineStage

A single backend stage in the OCR pipeline.

Field	Type	Default	Description
`backend`	`String`	—	Backend name: “tesseract”, “paddleocr”, “paddle-ocr”, “sceptre”, “vlm”, or a custom registered name. Sceptre uses ONNX Runtime on desktop/server and tract on Android/iOS; browser WebAssembly has a separate byte-fed engine because the normal async OCR registry assumes native model storage.
`priority`	`u32`	`100`	Priority weight (higher = tried first). Stages are sorted by priority descending.
`language`	`Option<Vec<String>>`	`/* serde(default) */`	Language override for this stage (None = use parent OcrConfig.language). A list is the canonical form and the only form accepted by the binding object APIs: `["eng", "deu"]`. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”).
`tesseract_config`	`Option<TesseractConfig>`	`/* serde(default) */`	Tesseract-specific config override for this stage.
`paddle_ocr_config`	`Option<serde_json::Value>`	`/* serde(default) */`	Legacy PaddleOCR config JSON for this stage. Deprecated since 1.3.2 and planned for removal in 2.0. Use `Self::paddle_ocr_settings` for typed access.
`paddle_ocr_settings`	`Option<PaddleOcrConfig>`	`/* serde(default) */`	Typed PaddleOCR config for this stage. When both PaddleOCR fields are set, this typed field takes precedence.
`vlm_config`	`Option<LlmConfig>`	`/* serde(default) */`	VLM config override for this pipeline stage.
`backend_options`	`Option<serde_json::Value>`	`/* serde(default) */`	Arbitrary per-call options passed through to the backend unchanged. Backends that support runtime tuning (mode switching, preprocessing flags, inference parameters, etc.) read this value and deserialize the keys they care about. Keys unknown to the backend are silently ignored, so options from different backends can coexist in the same config without conflict. Example (custom backend): `{ "mode": "fast", "enable_layout": true }`

OcrPoint

A point in OCR raster pixel coordinates.

Field	Type	Default	Description
`x`	`u32`	—	Horizontal coordinate.
`y`	`u32`	—	Vertical coordinate.

OcrProcessor

Tesseract-backed OCR processor with persistent result caching.

OcrQualityThresholds

Quality thresholds for OCR fallback decisions and pipeline quality gating.

Fields default to conservative extraction behavior. Suspected OCR recognition noise is reported but retained unless destructive filtering is explicitly enabled.

Field	Type	Default	Description
`min_total_non_whitespace`	`usize`	`64`	Minimum total non-whitespace characters to consider text substantive.
`min_non_whitespace_per_page`	`f64`	`32`	Minimum non-whitespace characters per page on average.
`min_meaningful_word_len`	`usize`	`4`	Minimum character count for a word to be “meaningful”.
`min_meaningful_words`	`usize`	`3`	Minimum count of meaningful words before text is accepted.
`min_alnum_ratio`	`f64`	`0.3`	Minimum alphanumeric ratio (non-whitespace chars that are alphanumeric).
`min_garbage_chars`	`usize`	`5`	Minimum Unicode replacement characters (U+FFFD) to trigger OCR fallback.
`max_fragmented_word_ratio`	`f64`	`0.6`	Maximum fraction of short (1-2 char) words before text is considered fragmented.
`critical_fragmented_word_ratio`	`f64`	`0.8`	Critical fragmentation threshold — triggers OCR regardless of meaningful words. Normal English text has ~20-30% short words. 80%+ is definitive garbage.
`max_ocr_output_fragmented_word_ratio`	`f64`	`0.35`	Maximum fraction of short (1-2 char) words an OCR result may carry before the page is reported as suspected recognition noise. This is a different decision, and a different operating point, from `Self::max_fragmented_word_ratio` / `Self::critical_fragmented_word_ratio`: those ask “is the native text bad enough that we should OCR this page?”, and are tuned to only fire on text that is already definitively broken (0.6 / 0.8). Here we are judging what OCR produced, where the failure mode is an engine run over line art — a scanned plat, an engineering drawing, a signature flourish — returning confident-looking strings that are not words. Measured over the 16 pages of a recorded municipal ordinance (13 prose pages, 3 scanned survey drawings): prose ran 0.04-0.28, the drawings 0.42-0.47. The default sits in that gap with margin on both sides. By default, crossing the threshold emits a processing warning but retains the recognized text. Set `Self::discard_suspected_ocr_noise` to `true` to restore destructive filtering.
`min_ocr_mean_confidence`	`f64`	`75`	Minimum mean OCR confidence (0-100) below which a page is reported as suspected noise. This is the engine’s own uncertainty about what it read, and it is a far sharper instrument than any statistic derived from the output text. Measured per page over a recorded municipal ordinance (10 prose pages, 6 scanned survey/architectural drawings) with Tesseract 5.5.3: `prose 86.3 - 95.3 drawings 18.5 - 64.3` The default sits in that gap with ~11 points of margin on each side. Compare the short-word ratio, which separated the same two groups by 0.09 on a 0-1 scale. A backend that reports no confidence (no `mean_text_conf` in its result metadata) skips this check entirely and falls back to `Self::max_ocr_output_fragmented_word_ratio`. By default, the signal emits a warning without discarding content. Set to 0.0 to disable the signal.
`min_words_for_ocr_output_check`	`usize`	`20`	Minimum word count before `Self::max_ocr_output_fragmented_word_ratio` may report a page. Short pages (a signature block, an exhibit title) are legitimately dominated by short words, so the ratio is not meaningful on them and the signal stays disabled.
`max_ocr_output_dict_invalid_word_ratio`	`f64`	`1.01`	Maximum fraction of a Tesseract page’s dictionary-checkable words that `TesseractAPI::is_valid_word` may reject before the page is treated as recognition noise, supplementing (never replacing) `Self::max_ocr_output_fragmented_word_ratio`. Tesseract-only: other backends never report this ratio (see `dictionary_invalid_word_ratio` in `ocr::processor::execution`), so this threshold is simply never consulted for them. Calibration owed, same as `ocr::types::TesseractConfig::min_confidence`: unlike `Self::max_ocr_output_fragmented_word_ratio` (measured across a recorded ordinance’s prose vs. drawing pages), this ratio has not yet had a page-level measurement run over a labeled corpus. The default therefore disables the check entirely (`1.01`, above the `[0.0, 1.0]` range a ratio can reach) rather than guess an operating point. Do not lower this without running that measurement first.
`discard_suspected_ocr_noise`	`bool`	`false`	Discard non-empty OCR text when any configured recognition-noise signal fires. Defaults to `false`: suspected noise is retained and surfaced through `processing_warnings`. Set to `true` to preserve the legacy destructive behavior.
`min_avg_word_length`	`f64`	`2`	Minimum average word length. Below this with enough words indicates garbled extraction.
`min_words_for_avg_length_check`	`usize`	`50`	Minimum word count before average word length check applies.
`min_consecutive_repeat_ratio`	`f64`	`0.08`	Minimum consecutive word repetition ratio to detect column scrambling.
`min_words_for_repeat_check`	`usize`	`50`	Minimum word count before consecutive repetition check is applied.
`substantive_min_chars`	`usize`	`100`	Minimum character count for “substantive markdown” OCR skip gate.
`non_text_min_chars`	`usize`	`20`	Minimum character count for “non-text content” OCR skip gate.
`alnum_ws_ratio_threshold`	`f64`	`0.4`	Alphanumeric+whitespace ratio threshold for skip decisions.
`pipeline_min_quality`	`f64`	`0.5`	Minimum quality score (0.0-1.0) for a pipeline stage result to be accepted. If the result from a backend scores below this, try the next backend.
`min_undecodable_ratio`	`f64`	`0.5`	Minimum fraction of non-whitespace characters that are undecodable (Unicode Private Use Area, replacement characters, or non-whitespace control characters) before a page’s text layer is treated as unreadable and routed to OCR (issue #1254). Gated by `min_total_non_whitespace` so short snippets with a stray symbol or two do not trip this check.
`enable_provenance_ocr_routing`	`bool`	`true`	Whether to route a page to OCR when xberg_native_pdf reports that a high fraction of its text was fabricated rather than read from the file (`MappingProvenance::Fallback`, xberg_native_pdf 0.3.75+, issue #1254). This is a direct fact from the extractor’s ISO 32000-1 §9.10.2 mapping cascade, distinct from the character-heuristic proxy behind `min_undecodable_ratio`. Defaults to `true`.
`min_provenance_fallback_ratio`	`f64`	`0.5`	Minimum fraction of a page’s non-whitespace characters with `MappingProvenance::Fallback` provenance before the page is treated as having a fabricated text layer and routed to OCR (issue #1254). Gated by `min_total_non_whitespace` so a short page with a stray fallback character cannot trip it. Only used when `enable_provenance_ocr_routing` is `true`.
`enable_plausibility_ocr_routing`	`bool`	`true`	Whether to route a page to OCR when its decoded text does not read as any real, detectable language (issue #1696). Unlike `enable_provenance_ocr_routing`, this catches a `/ToUnicode` CMap that resolves every glyph to a character, but consistently the WRONG one (e.g. a ROT-shifted mapping) – text that is structurally indistinguishable from real prose to every character-shape check, including the provenance signal, because the mapping tier really is file-backed. Defaults to `true`.
`min_reliable_language_chunk_ratio`	`f64`	`0.1`	Minimum fraction of a page’s language-detection chunks that whatlang classifies as reliable (`Info::is_reliable()`) before the page is trusted as legible (issue #1696). Below this AND below the mean-confidence guard together, the page’s text layer is treated as implausible and routed to OCR. Only used when `enable_plausibility_ocr_routing` is `true`.

Methods

default()

Signature:

pub fn default() -> OcrQualityThresholds

Example:

let result = OcrQualityThresholds::default();

Returns: OcrQualityThresholds

OcrRotation

Rotation information for an OCR element.

Field	Type	Default	Description
`angle_degrees`	`f64`	—	Rotation angle in degrees (0, 90, 180, 270 for PaddleOCR).
`confidence`	`Option<f64>`	`None`	Confidence score for the rotation detection.

OcrTable

Table detected via OCR.

Represents a table structure recognized during OCR processing.

Field	Type	Default	Description
`cells`	`Vec<Vec<String>>`	—	Table cells as a 2D vector (rows × columns)
`markdown`	`String`	—	Markdown representation of the table
`page_number`	`u32`	—	Page number where the table was found (1-indexed)
`bounding_box`	`Option<OcrTableBoundingBox>`	`/* serde(default) */`	Bounding box of the table in pixel coordinates (from OCR word positions).

OcrTableBoundingBox

Bounding box for an OCR-detected table in pixel coordinates.

Field	Type	Default	Description
`left`	`u32`	—	Left x-coordinate (pixels)
`top`	`u32`	—	Top y-coordinate (pixels)
`right`	`u32`	—	Right x-coordinate (pixels)
`bottom`	`u32`	—	Bottom y-coordinate (pixels)

OcrTesseractConfig

Configuration for Tesseract OCR (internal, efficient types).

This is the internal representation used by the OCR processor. Public API uses i32 for PyO3 compatibility, converted to u8 here for efficiency.

Field	Type	Default	Description
`language`	`String`	`"eng"`	Tesseract language code (e.g. `"eng"`, `"deu"`, `"eng+fra"`).
`psm`	`u8`	`3`	Page Segmentation Mode as a raw `u8` (see `PSMMode`).
`output_format`	`String`	`"markdown"`	OCR output format: `"text"`, `"markdown"`, `"hocr"`, or `"tsv"`.
`oem`	`u8`	`3`	OCR Engine Mode (0 = Legacy, 1 = LSTM, 2 = Both, 3 = Default).
`min_confidence`	`f64`	`0`	Minimum word confidence threshold (0.0–100.0); words below are dropped. Applied per WORD, not per page (see `parse_tsv_to_elements` and `extract_elements_via_iterator` in `ocr::processor::execution`): an individual low-confidence word is dropped from the output while the rest of the page’s words are kept. A floor set too high can still empty an entire page one word at a time, which is observably identical to a page-level drop, so this is not a “safe by construction” knob. See `Self::default` for why this stays at `0.0` and what measurement is owed before raising it.
`preprocessing`	`Option<ImagePreprocessingConfig>`	`None`	Optional image preprocessing applied before recognition.
`enable_table_detection`	`bool`	`true`	Whether to attempt table detection from hOCR/TSV output.
`table_min_confidence`	`f64`	`0`	Minimum confidence for cells included in reconstructed tables.
`table_column_threshold`	`u32`	`50`	Pixel threshold for grouping words into columns.
`table_row_threshold_ratio`	`f64`	`0.5`	Fraction of column height a word must span to count as a row separator.
`use_cache`	`bool`	`true`	Whether to use the on-disk OCR result cache.
`classify_use_pre_adapted_templates`	`bool`	`true`	Tesseract `classify_use_pre_adapted_templates` variable.
`language_model_ngram_on`	`bool`	`true`	Tesseract `language_model_ngram_on` variable. Enables Tesseract’s character n-gram language model, which penalizes output that does not look like a word of the target language even when the classifier itself was confident about individual glyphs. This is the dominant failure mode this library has measured for scanned line art (survey plats, engineering drawings, signature flourishes): the engine reads confident-looking non-words such as `LAAALDLI` that downstream heuristics only partially catch (see `is_ocr_recognition_noise` in `extractors::pdf::ocr`). Left off, Tesseract does not apply this penalty at all. Kept on by default rather than off, unlike upstream’s own default, because prose pages are unaffected while noise pages are meaningfully suppressed. This default must stay in sync with `language_model_ngram_on`’s default (the public-facing counterpart, in `types/formats.rs`). The two structs have independent `Default` impls, and several call sites in `extractors::image` construct the public struct’s default and convert it (via the `From` impl below) before this default is ever consulted — so this field’s value only reaches standalone image OCR when both defaults agree.
`tessedit_dont_blkrej_good_wds`	`bool`	`true`	Tesseract `tessedit_dont_blkrej_good_wds` variable.
`tessedit_dont_rowrej_good_wds`	`bool`	`true`	Tesseract `tessedit_dont_rowrej_good_wds` variable.
`tessedit_enable_dict_correction`	`bool`	`true`	Tesseract `tessedit_enable_dict_correction` variable.
`tessedit_char_whitelist`	`String`	`""`	Restrict recognized characters to this set (empty = unrestricted).
`tessedit_char_blacklist`	`String`	`""`	Exclude these characters from recognition (empty = none excluded).
`tessedit_use_primary_params_model`	`bool`	`true`	Tesseract `tessedit_use_primary_params_model` variable.
`textord_space_size_is_variable`	`bool`	`true`	Tesseract `textord_space_size_is_variable` variable.
`thresholding_method`	`u8`	`0`	Tesseract `thresholding_method` engine variable (0-2): 0 = Otsu, 1 = LeptonicaOtsu, 2 = Sauvola. Sent to Tesseract as a decimal integer string — see GH#1784: this used to be a `bool` sent as `"true"`/`"false"`, which Tesseract’s integer parameter parser (`stream >> intval`) silently failed to read, so the setting had no effect.
`auto_rotate`	`bool`	`false`	Enable automatic page rotation based on orientation detection. When enabled (and the `auto-rotate` build feature is compiled in), page orientation (0/90/180/270 degrees) is detected with an ONNX PP-LCNet document-orientation classifier — NOT Tesseract’s own `DetectOrientationScript()`/OSD, which this library does not call. The classifier reports only a rotation angle and confidence; it has no script-detection capability, so no script name is produced. If the page is rotated with high confidence, the image is corrected before recognition.
`tessdata_path`	`Option<PathBuf>`	`None`	Highest-priority override for the tessdata directory. When set, `resolve_tessdata_path` uses this path before consulting `TESSDATA_PREFIX`, cache, or system locations.
`source_dpi`	`Option<f64>`	`None`	True resolution, in DPI, of the image bytes accompanying this config, when the caller knows it. This is not a user-facing knob and has no counterpart on the public `TesseractConfig`: it is a per-call fact about one image, set by the PDF OCR route from `SOURCE_DPI_BACKEND_OPTION` because that route rendered the page and can derive it exactly. Every other caller leaves it `None`. `None` means “unknown”, and the DPI-normalization step then falls back to assuming 72 DPI as it always has. That assumption is what this field exists to displace: a page rendered at 150 DPI but declared as 72 gets scaled by `target_dpi / 72` instead of `target_dpi / 150`, which on a Letter page means upscaling into the 4096px dimension clamp for no added information and then telling Tesseract a `scan_res` that is roughly half the raster’s real resolution.
`known_full_page_scan`	`bool`	`false`	Whether the caller already knows this image is a whole-page scan, when true. Like `source_dpi`, this is not a user-facing knob and has no counterpart on the public `TesseractConfig`: it is a per-call fact about one image, set by the PDF OCR route from `KNOWN_FULL_PAGE_SCAN_BACKEND_OPTION` because that route already runs the scan-detection density check to decide the page’s default PSM. Every other caller leaves it `false`. `false` means “unknown, judge from the pixels”: `prepare_ocr_image` then falls back to `should_apply_default_preprocessing`’s brightness heuristic, as it always has. `true` skips that heuristic and applies the default preprocessing (resample, binarisation, deskew) unconditionally – a known scan page must not lose it just because shaded table rows or a grey background make it read as dark (GH#1894).
`page_number`	`u32`	`1`	The true, 1-indexed page number of the source document that `image_bytes` came from, when the caller knows it. Like `source_dpi`, this is not a user-facing knob and has no counterpart on the public `TesseractConfig`: `perform_ocr` runs Tesseract on exactly one loaded image per call, so Tesseract’s own per-call page index (hOCR’s `ppageno`, the TSV `page_num` column, and the iterator extraction’s page argument) is always `0`/`1` regardless of which page of the source document this image actually is. Every parsed element, table, and `OcrElement` is stamped with this field’s value instead of that per-call index, so a caller processing page 2 of a multi-page document must set this to `2` for the resulting elements to report the right page. Callers that don’t know (or don’t need) the true page number leave it at the default, `1`.
`security_limits`	`Option<SecurityLimits>`	`None`	Security limits for the image decode this OCR call performs. Carried from `OcrConfig::security_limits`, which the caller’s `ExtractionConfig` populates before dispatch. Before GH#1651 this type had no such field, so `config_to_tesseract` had nothing to copy into and every Tesseract decode ran under `SecurityLimits::default()` on every route – a caller who raised the limit to admit a large scan was still refused at 100 MiB. `#[serde(skip)]` because it is injected at runtime and never read from a config file. These ARE part of the OCR cache key (`hash_security_limits` in `ocr::processor::config`), and must stay there. This doc previously argued the opposite – that limits only gate whether a decode is attempted and never change the text Tesseract produces, so hashing them would split entries identical in content. The first half is true and the conclusion still does not follow: the gate lives inside `perform_ocr`, which runs only on a cache MISS, so a request carrying a strict limit was served an earlier permissive request’s cached result and the limit never applied at all. That is a silent policy bypass, and it made `issue_1651_ocr_security_limits` flaky – whichever sibling test populated the entry first decided the outcome. Ordinary callers share one default `SecurityLimits`, so cache reuse is unaffected in practice. `None` means `SecurityLimits::default()`, never “disable the check”.

Methods

default()

Signature:

pub fn default() -> TesseractConfig

Example:

let result = OcrTesseractConfig::default();

Returns: TesseractConfig

OdpExtractor

Native Rust extractor for OpenDocument Presentation (.odp) files.

Methods

default()

Signature:

pub fn default() -> OdpExtractor

Example:

let result = OdpExtractor::default();

Returns: OdpExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

OdtExtractor

High-performance ODT extractor using native Rust XML parsing.

This extractor provides:

Fast text extraction via roxmltree XML parsing
Comprehensive metadata extraction from meta.xml
Table extraction with row and cell support
Formatting preservation (bold, italic, strikeout)
Support for headings, paragraphs, and special elements

Methods

default()

Signature:

pub fn default() -> OdtExtractor

Example:

let result = OdtExtractor::default();

Returns: OdtExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

OdtProperties

OpenDocument metadata from meta.xml

Contains metadata fields defined by the OASIS OpenDocument Format standard. Uses Dublin Core elements (dc:) and OpenDocument meta elements (meta:).

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Document title (dc:title)
`subject`	`Option<String>`	`Default::default()`	Document subject/topic (dc:subject)
`creator`	`Option<String>`	`Default::default()`	Current document creator/author (dc:creator)
`initial_creator`	`Option<String>`	`Default::default()`	Initial creator of the document (meta:initial-creator)
`keywords`	`Option<String>`	`Default::default()`	Keywords or tags (meta:keyword)
`description`	`Option<String>`	`Default::default()`	Document description (dc:description)
`date`	`Option<String>`	`Default::default()`	Current modification date (dc:date)
`creation_date`	`Option<String>`	`Default::default()`	Initial creation date (meta:creation-date)
`language`	`Option<String>`	`Default::default()`	Document language (dc:language)
`generator`	`Option<String>`	`Default::default()`	Generator/application that created the document (meta:generator)
`editing_duration`	`Option<String>`	`Default::default()`	Editing duration in ISO 8601 format (meta:editing-duration)
`editing_cycles`	`Option<String>`	`Default::default()`	Number of edits/revisions (meta:editing-cycles)
`page_count`	`Option<i32>`	`Default::default()`	Document statistics - page count (meta:page-count)
`word_count`	`Option<i32>`	`Default::default()`	Document statistics - word count (meta:word-count)
`character_count`	`Option<i32>`	`Default::default()`	Document statistics - character count (meta:character-count)
`paragraph_count`	`Option<i32>`	`Default::default()`	Document statistics - paragraph count (meta:paragraph-count)
`table_count`	`Option<i32>`	`Default::default()`	Document statistics - table count (meta:table-count)
`image_count`	`Option<i32>`	`Default::default()`	Document statistics - image count (meta:image-count)

OpenWebDocumentMetadata

Metadata for the OpenWebUI external document loader response.

Field	Type	Default	Description
`source`	`String`	—	Original filename

OpenWebDocumentResponse

OpenWebUI “External” engine response format.

Returned by PUT /process for the OpenWebUI external document loader.

Field	Type	Default	Description
`page_content`	`String`	—	Extracted text content
`metadata`	`OpenWebDocumentMetadata`	—	Document metadata

OpmlExtractor

OPML format extractor.

Extracts outline structure and metadata from OPML documents using native Rust parsing.

Methods

default()

Signature:

pub fn default() -> OpmlExtractor

Example:

let result = OpmlExtractor::default();

Returns: OpmlExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

OrgModeExtractor

Methods

default()

Signature:

pub fn default() -> OrgModeExtractor

Example:

let result = OrgModeExtractor::default();

Returns: OrgModeExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

OrientationResult

Document orientation detection result.

Field	Type	Default	Description
`degrees`	`u32`	—	Detected orientation in degrees (0, 90, 180, or 270).
`confidence`	`f32`	—	Confidence score (0.0-1.0).

OrtSessionOptions

Process-wide memory and threading options applied to the ONNX Runtime sessions of embeddings, reranking, sparse embeddings and late-interaction engines. Rust API only.

Other ORT users (layout detection, SLANet, TATR, the inference backend, Whisper, paddle OCR, GLiNER) build their own sessions and are not affected.

The defaults reproduce the historical behaviour: ORT’s memory-pattern planner and CPU arena stay enabled and the intra-op thread count follows the default thread budget. Both ORT features retain allocations sized by the largest batch seen, which is what makes resident memory grow with variable-length embedding batches and never shrink. Turning them off trades some throughput for a bounded footprint.

Options are read when a session is built. set_ort_session_options clears the engine caches when the options change, so resident engines are rebuilt on next use; a caller still holding an engine keeps its old session until it drops it.

Field	Type	Default	Description
`memory_pattern`	`bool`	`true`	Enable ORT’s memory-pattern optimization. Disable for variable input sizes. Default `true`.
`cpu_arena`	`bool`	`true`	Enable the CPU execution provider’s arena allocator. Disable so freed tensors return to the system allocator instead of being retained. Applies to the CPU execution provider only; CUDA/TensorRT device arenas are unaffected. Default `true`.
`max_threads`	`Option<usize>`	`None`	Intra-op thread count. `None` (default) uses the standard thread budget (`min(cores, 8)` or the cgroup quota). Values below 1 are raised to 1. Only the ORT intra-op pool is changed; `EMBED_SEMAPHORE` and other `resolve_thread_budget` callers are unaffected. ONNX Runtime builds that use OpenMP ignore this setting; configure `OMP_NUM_THREADS` for those builds.

Methods

default()

Signature:

pub fn default() -> OrtSessionOptions

Example:

let result = OrtSessionOptions::default();

Returns: OrtSessionOptions

PaddleOcrBackend

PaddleOCR backend using ONNX Runtime.

Maintains a pool of OCR engines keyed by script family. Each family has its own recognition model and character dictionary, while detection and classification models are shared across all families.

Thread Safety

The backend is Send + Sync and can be used across threads safely via Arc. Per-key initialization cells serialize each cold start. Initialized engines can run OCR concurrently without holding the pool lock.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

confidence_semantics()

PaddleOCR never derives a page-level aggregate confidence: per-element text_score exists but no page-level number is written.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

page_orientation_handling()

Measured on a /Rotate 270 scanned ordinance: PaddleOCR warps each detected min-area-rect quad upright before recognition, so the recognised text is correct, but the block list stays in raw raster (y, x) order — the caller must reorder blocks itself.

Signature:

pub fn page_orientation_handling(&self) -> PageOrientationHandling

Example:

let result = instance.page_orientation_handling();

Returns: PageOrientationHandling

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

supports_table_detection()

Signature:

pub fn supports_table_detection(&self) -> bool

Example:

let result = instance.supports_table_detection();

Returns: bool

default()

Signature:

pub fn default() -> PaddleOcrBackend

Example:

let result = PaddleOcrBackend::default();

Returns: PaddleOcrBackend

PaddleOcrConfig

Configuration for PaddleOCR backend.

Configures PaddleOCR text detection and recognition with multi-language support. Uses a builder pattern for convenient configuration.

Field	Type	Default	Description
`language`	`String`	`"en"`	Language code (e.g., “en”, “ch”, “jpn”, “kor”, “deu”, “fra”)
`cache_dir`	`Option<PathBuf>`	`None`	Optional Hugging Face Hub cache root for model files. When unset, `XBERG_OCR_MODEL_CACHE_DIR` is used when non-empty, followed by the standard `HF_HUB_CACHE`, legacy `HUGGINGFACE_HUB_CACHE`, and `HF_HOME` conventions.
`use_angle_cls`	`bool`	`false`	Enable angle classification for rotated text (default: false). Can misfire on short text regions, rotating crops incorrectly before recognition.
`enable_table_detection`	`bool`	`false`	Enable table structure detection (default: false, unlike `TesseractConfig` which defaults to `true` — see `crate::ocr::types::TesseractConfig`). PaddleOCR has no per-word table-candidate confidence carve-out the way Tesseract’s TSV does (every recognised word is a clustering candidate), so unfiltered clustering over-fabricates tables on ordinary prose. This stays off until it clears a cell-scored accuracy measurement (see the pinning test at the bottom of this file). Consequence for callers: switching an existing config from the Tesseract backend to PaddleOCR with stock defaults silently produces no OCR tables — enable this explicitly if you need them. When enabled, this only clusters and reconstructs a grid from word boxes PaddleOCR has already produced (see `crate::paddle_ocr::backend::PaddleOcrBackend::process_image`) — it does not run any additional model inference, so the added per-page cost is the CPU-only clustering/reconstruction pass, not another ONNX call.
`det_db_thresh`	`f32`	`0.3`	Database threshold for text detection (default: 0.3) Range: 0.0-1.0, higher values require more confident detections
`det_db_box_thresh`	`f32`	`0.5`	Box threshold for text bounding box refinement (default: 0.5) Range: 0.0-1.0
`det_db_unclip_ratio`	`f32`	`1.6`	Unclip ratio for expanding text bounding boxes (default: 1.6) Controls the expansion of detected text regions
`det_limit_side_len`	`u32`	`1024`	Maximum side length for detection image (default: 1024) Larger images may be resized to this limit for faster inference
`rec_batch_num`	`u32`	`6`	Batch size for recognition inference (default: 6) Number of text regions to process simultaneously
`padding`	`u32`	`10`	Padding in pixels added around the image before detection (default: 10). Large values can include surrounding content like table gridlines.
`drop_score`	`f32`	`0.5`	Minimum recognition confidence score for text lines (default: 0.5). Text regions with recognition confidence below this threshold are discarded. Matches PaddleOCR Python’s `drop_score` parameter. Range: 0.0-1.0
`model_tier`	`String`	`"mobile"`	Model tier controlling detection/recognition model size and accuracy trade-off. For PP-OCRv5 (`model_version = "pp-ocrv5"`): - `"mobile"` (default): Lightweight models (~4.5MB detection, ~16.5MB recognition), fast download and inference - `"server"`: Large, high-accuracy models (~88MB detection, ~84MB recognition), best for GPU or complex documents For PP-OCRv6 (`model_version = "pp-ocrv6"`, the default): - `"small"`: ~9.9MB detection, full 18,708-char CJK+Latin+JA/KO recognition dictionary. The default `"mobile"` resolves here, so this is what an unconfigured extraction uses. - `"medium"`: ~62MB detection, same dictionary. Higher accuracy, substantially slower on CPU. A legacy `"server"` tier, or any unrecognised value, resolves here. - `"tiny"`: ~1.8MB detection, but a reduced 6,904-char (~zh/en) dictionary — it cannot read the scripts the other two cover. Note PaddleOCR pages do not run concurrently: the ONNX session is held behind a mutex, so the thread budget goes to intra-op parallelism and wall time scales with page count times per-page inference. Tier choice therefore dominates throughput on multi-page documents.
`model_version`	`String`	`"pp-ocrv6"`	Model generation: `"pp-ocrv6"` (default) or `"pp-ocrv5"`. PP-OCRv6 adds a unified CJK+Latin+JA/KO recognition model with `medium`/`small`/`tiny` tiers (see `model_tier`). Scripts outside the v6 unified coverage (Arabic, Cyrillic, Devanagari, Greek, Tamil, Telugu, Thai) transparently fall back to the PP-OCRv5 per-script recognition models. Defaults to `"pp-ocrv6"`; the default `model_tier` (`"mobile"`) resolves to the v6 `"small"` tier. Select `"pp-ocrv5"` to pin the legacy per-script/unified fleet.
`inference_backend`	`Option<PaddleInferenceBackend>`	`None`	Explicit inference engine choice. `None` (the default) resolves to the compiled default: `ort` when the `paddle-ocr-ort` feature is compiled in, otherwise `tract`. An explicit choice is validated against the compiled features when the OCR engine is constructed (see `crate::paddle_ocr::backend::effective_backend`); requesting an engine whose feature is not compiled in is a clear configuration error rather than a silent fallback.

Methods

with_cache_dir()

Sets a custom Hugging Face Hub cache root for model files.

Signature:

pub fn with_cache_dir(self, path: PathBuf) -> PaddleOcrConfig

Example:

use xberg::PaddleOcrConfig;
use std::path::PathBuf;

let config = PaddleOcrConfig::new("en")
    .with_cache_dir(PathBuf::from("/tmp/paddle-cache"));

Parameters:

Name	Type	Required	Description
`path`	`PathBuf`	Yes	Path to cache directory

Returns: PaddleOcrConfig

with_table_detection()

Enables or disables table structure detection.

Signature:

pub fn with_table_detection(self, enable: bool) -> PaddleOcrConfig

Example:

use xberg::PaddleOcrConfig;

let config = PaddleOcrConfig::new("en")
    .with_table_detection(true);

Parameters:

Name	Type	Required	Description
`enable`	`bool`	Yes	Whether to enable table detection

Returns: PaddleOcrConfig

with_angle_cls()

Enables or disables angle classification for rotated text.

Signature:

pub fn with_angle_cls(self, enable: bool) -> PaddleOcrConfig

Example:

let result = instance.with_angle_cls(true);

Parameters:

Name	Type	Required	Description
`enable`	`bool`	Yes	Whether to enable angle classification

Returns: PaddleOcrConfig

with_det_db_thresh()

Sets the database threshold for text detection.

Signature:

pub fn with_det_db_thresh(self, threshold: f32) -> PaddleOcrConfig

Example:

let result = instance.with_det_db_thresh(0.5);

Parameters:

Name	Type	Required	Description
`threshold`	`f32`	Yes	Detection threshold (0.0-1.0)

Returns: PaddleOcrConfig

with_det_db_box_thresh()

Sets the box threshold for text bounding box refinement.

Signature:

pub fn with_det_db_box_thresh(self, threshold: f32) -> PaddleOcrConfig

Example:

let result = instance.with_det_db_box_thresh(0.5);

Parameters:

Name	Type	Required	Description
`threshold`	`f32`	Yes	Box threshold (0.0-1.0)

Returns: PaddleOcrConfig

with_det_db_unclip_ratio()

Sets the unclip ratio for expanding text bounding boxes.

Signature:

pub fn with_det_db_unclip_ratio(self, ratio: f32) -> PaddleOcrConfig

Example:

let result = instance.with_det_db_unclip_ratio(0.5);

Parameters:

Name	Type	Required	Description
`ratio`	`f32`	Yes	Unclip ratio (typically 1.5-2.0)

Returns: PaddleOcrConfig

with_det_limit_side_len()

Sets the maximum side length for detection images.

Signature:

pub fn with_det_limit_side_len(self, length: u32) -> PaddleOcrConfig

Example:

let result = instance.with_det_limit_side_len(42);

Parameters:

Name	Type	Required	Description
`length`	`u32`	Yes	Maximum side length in pixels

Returns: PaddleOcrConfig

with_rec_batch_num()

Sets the batch size for recognition inference.

Signature:

pub fn with_rec_batch_num(self, batch_size: u32) -> PaddleOcrConfig

Example:

let result = instance.with_rec_batch_num(42);

Parameters:

Name	Type	Required	Description
`batch_size`	`u32`	Yes	Number of text regions to process simultaneously

Returns: PaddleOcrConfig

with_drop_score()

Sets the minimum recognition confidence threshold.

Signature:

pub fn with_drop_score(self, score: f32) -> PaddleOcrConfig

Example:

let result = instance.with_drop_score(0.5);

Parameters:

Name	Type	Required	Description
`score`	`f32`	Yes	Minimum confidence (0.0-1.0), text below this is dropped

Returns: PaddleOcrConfig

with_padding()

Sets padding in pixels added around images before detection.

Signature:

pub fn with_padding(self, padding: u32) -> PaddleOcrConfig

Example:

let result = instance.with_padding(42);

Parameters:

Name	Type	Required	Description
`padding`	`u32`	Yes	Padding in pixels (0-100)

Returns: PaddleOcrConfig

with_model_tier()

Sets the model tier controlling detection/recognition model size.

Signature:

pub fn with_model_tier(self, tier: String) -> PaddleOcrConfig

Example:

let result = instance.with_model_tier("value");

Parameters:

Name	Type	Required	Description
`tier`	`String`	Yes	`"mobile"` (default, lightweight, faster) or `"server"` (high accuracy, GPU/complex documents)

Returns: PaddleOcrConfig

with_model_version()

Sets the model generation.

Signature:

pub fn with_model_version(self, version: String) -> PaddleOcrConfig

Example:

let result = instance.with_model_version("value");

Parameters:

Name	Type	Required	Description
`version`	`String`	Yes	`"pp-ocrv6"` (default) or `"pp-ocrv5"`. Under `"pp-ocrv6"`, `model_tier` selects among `"medium"`/`"small"`/`"tiny"`.

Returns: PaddleOcrConfig

default()

Creates a default configuration with English language support.

Signature:

pub fn default() -> PaddleOcrConfig

Example:

let result = PaddleOcrConfig::default();

Returns: PaddleOcrConfig

PaddleOcrVlBackend

PaddleOCR-VL backend using candle transformers.

A vision-language model for comprehensive document parsing. Supports text recognition, tables, formulas, and charts through a unified interface with markdown output. The engine implements the PaddleOCR-VL 1.5 architecture (SigLIP vision encoder + ERNIE 4.5 text decoder); PaddleOCR-VL 1.6 keeps that architecture unchanged, so it loads through the same engine — only the default weights changed.

Supports 109+ languages through the PaddlePaddle pretrained models.

Configuration

PaddleOCR-VL accepts backend options for task selection, device, and weight source:

{
  "task": "ocr",
  "device": "auto",
  "model_id": "xberg-io/paddleocr-vl-1.6",
  "model_path": "/path/to/paddleocr-vl-model"
}

task (string, optional): per-call override for "ocr", "table", "formula", or "chart". When omitted, the task selected when constructing the backend is used.
device (string): "auto", "cpu", "cuda", "metal"
model_id (string): HuggingFace repo id to auto-download weights from. Defaults to xberg-io/paddleocr-vl-1.6, a checksum-pinned mirror of PaddlePaddle/PaddleOCR-VL-1.6. Ignored when model_path is set.
model_path (string, optional): path to a local model directory. When omitted, the weights named by model_id are downloaded on first use into the standard Hugging Face cache — no manual staging required.
hf_revision (string, optional): immutable commit for a custom model_id. The default model is pinned automatically.
cache_dir (string, optional): explicit Hugging Face Hub cache root. When omitted, HF_HUB_CACHE, HUGGINGFACE_HUB_CACHE, and HF_HOME are honored.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Process an image using the PaddleOCR-VL engine.

Errors:

Returns Validation if image_bytes is empty. Returns Ocr if weight download, device selection, engine initialisation, or inference fails.

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Process an image file using the PaddleOCR-VL engine.

Errors:

Returns an error if the file cannot be read or if inference fails.

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

emits_structured_markdown()

Signature:

pub fn emits_structured_markdown(&self) -> bool

Example:

let result = instance.emits_structured_markdown();

Returns: bool

confidence_semantics()

PaddleOCR-VL reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

PaddleOcrVlBackendOptions

Runtime options accepted by the candle-paddleocr-vl backend.

Field	Type	Default	Description
`task`	`Option<PaddleOcrVlTaskKind>`	`Default::default()`	Optional per-call recognition task; the backend constructor’s task is used when absent.
`model_path`	`Option<String>`	`Default::default()`	Optional local model directory, which takes precedence over `model_id`.
`model_id`	`Option<String>`	`Default::default()`	Optional Hugging Face repository identifier.
`hf_revision`	`Option<String>`	`Default::default()`	Optional immutable Hugging Face model revision.
`cache_dir`	`Option<String>`	`Default::default()`	Optional Hugging Face cache root.
`device`	`Option<CandleDevicePreference>`	`Default::default()`	Optional per-call device override.

PageBoundary

Byte offset boundary for a page.

Tracks where a specific page’s content starts and ends in the main content string, enabling mapping from byte positions to page numbers. Offsets are guaranteed to be at valid UTF-8 character boundaries when using standard String methods (push_str, push, etc.).

Field	Type	Default	Description
`byte_start`	`usize`	—	Byte offset where this page starts in the content string (UTF-8 valid boundary, inclusive)
`byte_end`	`usize`	—	Byte offset where this page ends in the content string (UTF-8 valid boundary, exclusive)
`page_number`	`u32`	—	Page number (1-indexed)

PageClassification

Classification result for a single page.

Field	Type	Default	Description
`page_number`	`u32`	—	1-indexed page number this classification belongs to.
`labels`	`Vec<ClassificationLabel>`	—	Labels assigned to the page. Single-label classification yields exactly one entry; multi-label classification yields any subset of the configured label set.

PageClassificationConfig

Since: v1.0

Configuration for the page-classification post-processor.

Field	Type	Default	Description
`prompt_template`	`Option<String>`	`None`	Minijinja prompt template. Receives `{{ labels }}` (joined list), `{{ page_text }}` and `{{ multi_label }}` variables. `None` lets the backend pick a sensible default.
`labels`	`Vec<String>`	—	The set of labels the classifier may emit. Must contain at least one entry.
`multi_label`	`bool`	`/* serde(default) */`	Allow multiple labels per page. Single-label mode returns at most one label.
`llm`	`LlmConfig`	—	LLM configuration used for classification.

PageClassificationProcessor

Post-processor that asks an LLM to classify each page of the extracted content.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

PageConfig

Page extraction and tracking configuration.

Controls how pages are extracted, tracked, and represented in the extraction results. When None, page tracking is disabled.

Page range tracking in chunk metadata (first_page/last_page) is automatically enabled when page boundaries are available and chunking is configured.

Field	Type	Default	Description
`extract_pages`	`bool`	`false`	Extract pages as separate array (ExtractedDocument.pages)
`insert_page_markers`	`bool`	`false`	Insert page markers in main content string
`marker_format`	`String`	`"<!-- PAGE {page_num} -->"`	Page marker format (use {page_num} placeholder) Default: “\n\n\n\n”
`keep_native_content`	`bool`	`false`	Keep the text layer of a PDF page whose `content` OCR output replaces, for a caller that needs the exact characters of a short born-digital page. The text layer is returned in `PageContent::native_content`. Needs `extract_pages`.

Methods

default()

Signature:

pub fn default() -> PageConfig

Example:

let result = PageConfig::default();

Returns: PageConfig

PageContent

Content for a single page/slide.

When page extraction is enabled, documents are split into per-page content with associated tables and images mapped to each page.

Performance

Uses Arc-wrapped tables and images for memory efficiency:

Vec<Arc<Table>> enables zero-copy sharing of table data
Vec<Arc<ExtractedImage>> enables zero-copy sharing of image data
Maintains exact JSON compatibility via custom Serialize/Deserialize

This reduces memory overhead for documents with shared tables/images by avoiding redundant copies during serialization.

Field	Type	Default	Description
`page_number`	`u32`	—	Page number (1-indexed)
`content`	`String`	—	Text content for this page
`tables`	`Vec<Table>`	`/* serde(default) */`	Tables found on this page (uses Arc for memory efficiency) Serializes as `Vec<Table>` for JSON compatibility while maintaining Arc semantics in-memory for zero-copy sharing.
`image_indices`	`Vec<u32>`	`/* serde(default) */`	Indices into `ExtractedDocument.images` for images found on this page. Each value is a zero-based index into the top-level `images` collection. Only populated when `extract_images = true` in the extraction config.
`image_preprocessing`	`Option<ImagePreprocessingMetadata>`	`/* serde(default) */`	OCR image preprocessing applied to this page’s raster.
`hierarchy`	`Option<PageHierarchy>`	`None`	Hierarchy information for the page (when hierarchy extraction is enabled) Contains text hierarchy levels (H1-H6) extracted from the page content.
`is_blank`	`Option<bool>`	`None`	Whether this page is blank (no meaningful text content) Determined during extraction based on text content analysis. A page is blank if it has fewer than 3 non-whitespace characters and contains no tables or images.
`layout_regions`	`Option<Vec<LayoutRegion>>`	`None`	Layout detection regions for this page (when layout detection is enabled). Contains detected layout regions with class, confidence, bounding box, and area fraction. Only populated when layout detection is configured.
`speaker_notes`	`Option<String>`	`None`	Speaker notes for this slide (PPTX only). Contains the text from the slide’s notes pane (`ppt/notesSlides/notesSlide{N}.xml`). Only populated when the source is a PPTX file and notes are present.
`section_name`	`Option<String>`	`None`	Section name this slide belongs to (PPTX only). PowerPoint sections group slides into logical chapters (`<p:sectionLst>` in `ppt/presentation.xml`). Only populated when the source is a PPTX file and the slide belongs to a named section.
`sheet_name`	`Option<String>`	`None`	Sheet name for this page (XLSX/ODS only). Each spreadsheet sheet maps to one `PageContent` entry. This field carries the sheet’s display name as it appears in the workbook. `None` for all non-spreadsheet formats and for sheets with an empty name.
`ocr_confidence`	`Option<PageOcrConfidence>`	`/* serde(default) */`	Aggregate OCR confidence for this page. `None` when the page was not OCR’d.
`native_content`	`Option<String>`	`/* serde(default) */`	Text layer of this page, kept when OCR output replaced it in `content`. Set only when `PageConfig::keep_native_content` is on, an OCR route replaced this page’s `content`, and the text layer is not blank and differs from the OCR text. `None` on every other page. PDF only.

PageDimensions

Metadata for individual page/slide/sheet.

Captures per-page information including dimensions, content counts, and visibility state (for presentations).

Field	Type	Default	Description
`width`	`f64`	—	Page width in points or pixels.
`height`	`f64`	—	Page height in points or pixels.

PageHierarchy

Page hierarchy structure containing heading levels and block information.

Used when PDF text hierarchy extraction is enabled. Contains hierarchical blocks with heading levels (H1-H6) for semantic document structure.

Field	Type	Default	Description
`block_count`	`u32`	—	Number of hierarchy blocks on this page
`blocks`	`Vec<HierarchicalBlock>`	`/* serde(default) */`	Hierarchical blocks with heading levels

PageInfo

Field	Type	Default	Description
`number`	`u32`	—	Page number (1-indexed)
`title`	`Option<String>`	`None`	Page title (usually for presentations)
`dimensions`	`Option<PageDimensions>`	`/* serde(default) */`	Dimensions in points (PDF) or pixels (images).
`image_count`	`Option<u32>`	`None`	Number of images on this page
`table_count`	`Option<u32>`	`None`	Number of tables on this page
`hidden`	`Option<bool>`	`None`	Whether this page is hidden (e.g., in presentations)
`is_blank`	`Option<bool>`	`None`	Whether this page is blank (no meaningful text, no images, no tables) A page is considered blank if it has fewer than 3 non-whitespace characters and contains no tables or images. This is useful for filtering out empty pages in scanned documents or PDFs with blank separator pages.
`has_vector_graphics`	`bool`	`/* serde(default) */`	Whether this page contains non-trivial vector graphics (paths, shapes, curves) Indicates the presence of vector-drawn content such as charts, diagrams, or geometric shapes (e.g., from Adobe InDesign, LaTeX TikZ). These are invisible to `ExtractedDocument.images` since they are not embedded as raster XObjects. Set to `true` when path count exceeds a heuristic threshold, signaling that downstream consumers may want to rasterize the page to capture this content. Only populated for PDFs; `None` for other document types.

PageMargins

Page margins in twips (twentieths of a point).

Field	Type	Default	Description
`top`	`Option<i32>`	`Default::default()`	Top margin in twips.
`right`	`Option<i32>`	`Default::default()`	Right margin in twips.
`bottom`	`Option<i32>`	`Default::default()`	Bottom margin in twips.
`left`	`Option<i32>`	`Default::default()`	Left margin in twips.
`header`	`Option<i32>`	`Default::default()`	Header offset in twips.
`footer`	`Option<i32>`	`Default::default()`	Footer offset in twips.
`gutter`	`Option<i32>`	`Default::default()`	Gutter margin in twips.

PageMarginsPoints

Page margins converted to points (1/72 inch).

Field	Type	Default	Description
`top`	`Option<f64>`	`Default::default()`	Top margin in points.
`right`	`Option<f64>`	`Default::default()`	Right margin in points.
`bottom`	`Option<f64>`	`Default::default()`	Bottom margin in points.
`left`	`Option<f64>`	`Default::default()`	Left margin in points.
`header`	`Option<f64>`	`Default::default()`	Header offset in points.
`footer`	`Option<f64>`	`Default::default()`	Footer offset in points.
`gutter`	`Option<f64>`	`Default::default()`	Gutter margin in points.

PageOcrConfidence

Aggregate OCR legibility score for a page, reported by the backend that produced its text.

This is distinct from OcrConfidence, which scores a single detected element (a word or line) using detection/recognition confidence from the OCR engine itself. PageOcrConfidence is a page-level summary computed after noise filtering, intended for triage of which pages are worth a closer look, not for comparing OCR engines against each other.

Field	Type	Default	Description
`score`	`Option<f64>`	`Default::default()`	Aggregate legibility score in `0.0..=1.0`, or `None` when the backend that produced this page does not report a calibrated legibility scale. Backends differ in what their confidence numbers mean (see `OcrConfidence`’s per-backend normalization), and not every backend maps onto a 0.0-1.0 legibility scale at all. When a backend has no such calibrated scale, this is `None` rather than a misleading number, and scores must never be compared across backends.
`word_count`	`u32`	—	Number of words the score was averaged over, AFTER noise filtering. A small `word_count` means the average is based on little evidence, so a high `score` next to a small `word_count` is not representative of the whole page.
`backend`	`String`	—	Name of the OCR backend that produced the page text.

PageRange

Page range for a chunk (0-indexed, inclusive).

Field	Type	Default	Description
`start`	`u32`	—	Start page (0-indexed, inclusive).
`end`	`u32`	—	End page (0-indexed, inclusive).

Methods

page_count()

Get the number of pages in this range.

Signature:

pub fn page_count(&self) -> u32

Example:

let result = instance.page_count();

Returns: u32

PageSignals

Per-page signals extracted from PDF content.

Field	Type	Default	Description
`page_number`	`u32`	—	1-indexed page number.
`text_excerpt`	`String`	—	First ~500 characters of extracted text.
`starts_with_letterhead_like`	`bool`	—	`true` if page starts with letterhead-like content (ALL CAPS line in first 5 lines or a logo-image bbox at top).
`has_page_number_one_marker`	`bool`	—	`true` if text contains “Page 1” or “1 of N” pattern.
`has_signature_block`	`bool`	—	`true` if text contains signature indicators (“Sincerely”, “Signed”) or a signature image bbox.
`layout_text_density`	`f32`	—	Text density: characters per page area, normalised to `[0.0, 1.0]`.

Methods

from_page_text()

Derive signals from raw page text.

Callers that already have structured per-page data (e.g. from a PDF extractor) can set individual fields directly. This constructor is for callers that only have the plain-text content of a page (e.g. from PageContent).

Heuristics

All signal derivations are conservative starting points. Each is documented inline. They err on the side of fewer false positives; tune thresholds via MultidocThresholds rather than by changing these heuristics.

Signature:

pub fn from_page_text(page_number: u32, text: &str, layout_text_density: f32) -> PageSignals

Example:

let result = PageSignals::from_page_text(42, "value", 0.5);

Parameters:

Name	Type	Required	Description
`page_number`	`u32`	Yes	The page number
`text`	`&str`	Yes	The text
`layout_text_density`	`f32`	Yes	The layout text density

Returns: PageSignals

PageSpan

A single page covered by a chunk, with an optional bounding box on that page.

See ChunkMetadata::page_spans (#1295) for population semantics.

Field	Type	Default	Description
`page`	`u32`	—	Page number (1-indexed).
`bbox`	`Option<BoundingBox>`	`None`	Bounding box on this page, if known.

PageStructure

Unified page structure for documents.

Supports different page types (PDF pages, PPTX slides, Excel sheets) with character offset boundaries for chunk-to-page mapping.

Field	Type	Default	Description
`total_count`	`u32`	—	Total number of pages/slides/sheets
`unit_type`	`PageUnitType`	—	Type of paginated unit
`boundaries`	`Option<Vec<PageBoundary>>`	`None`	Character offset boundaries for each page Maps character ranges in the extracted content to page numbers. Used for chunk page range calculation.
`pages`	`Option<Vec<PageInfo>>`	`None`	Detailed per-page metadata (optional, only when needed)

PagesExtractor

Apple Pages document extractor.

Supports .pages files (modern iWork format, 2013+).

Extracts all text content from the document by parsing the IWA (iWork Archive) container: ZIP → Snappy → protobuf text fields.

Methods

default()

Signature:

pub fn default() -> PagesExtractor

Example:

let result = PagesExtractor::default();

Returns: PagesExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

ParaText

Plain text content decoded from a ParaText record (tag 0x43).

Field	Type	Default	Description
`content`	`String`	—	Decoded UTF-8 text with HWP control characters mapped to whitespace or removed. May contain `EQUATION_PLACEHOLDER` markers awaiting substitution.

Paragraph

A DOCX paragraph containing formatted text runs.

Field	Type	Default	Description
`runs`	`Vec<Run>`	`vec![]`	Formatted text runs that make up this paragraph.
`style`	`Option<String>`	`Default::default()`	Style ID applied to this paragraph (e.g. `"Heading1"`, `"Normal"`).
`numbering_id`	`Option<i64>`	`Default::default()`	Numbering definition ID for bulleted or numbered lists.
`numbering_level`	`Option<i64>`	`Default::default()`	Indentation level within the numbering definition (0-based).
`bookmarks`	`Vec<String>`	`vec![]`	Bookmark names (`w:bookmarkStart/@w:name`) that start inside this paragraph. A generated table of contents links each entry to the `_Toc…` bookmark Word writes into the heading paragraph, so these are the link targets internal `w:hyperlink w:anchor` references resolve against. Word’s `_GoBack` bookmark is filtered out — it records the last edit position, not a link target.
`in_table_of_contents`	`bool`	—	True when this paragraph is part of a table of contents. Set by either of the two markers Word emits: a `w:sdt` whose `w:sdtPr/w:docPartObj/w:docPartGallery` is `Table of Contents`, or a `TOC` field code.

ParagraphProperties

Paragraph-level formatting properties (alignment, spacing, indentation, etc.).

Field	Type	Default	Description
`alignment`	`Option<String>`	`Default::default()`	`"left"`, `"center"`, `"right"`, `"both"` (justified).
`spacing_before`	`Option<i32>`	`Default::default()`	Spacing before paragraph in twips.
`spacing_after`	`Option<i32>`	`Default::default()`	Spacing after paragraph in twips.
`spacing_line`	`Option<i32>`	`Default::default()`	Line spacing in twips or 240ths of a line.
`spacing_line_rule`	`Option<String>`	`Default::default()`	Line spacing rule: “auto”, “exact”, or “atLeast”.
`indent_left`	`Option<i32>`	`Default::default()`	Left indentation in twips.
`indent_right`	`Option<i32>`	`Default::default()`	Right indentation in twips.
`indent_first_line`	`Option<i32>`	`Default::default()`	First-line indentation in twips.
`indent_hanging`	`Option<i32>`	`Default::default()`	Hanging indentation in twips.
`outline_level`	`Option<u8>`	`Default::default()`	Outline level 0-8 for heading levels.
`numbering_id`	`Option<i64>`	`Default::default()`	Numbering definition id (`w:numPr/w:numId`) the style applies. Word lets a list style carry the numbering reference instead of putting it on each paragraph, which is what the built-in `List Bullet` and `List Number` styles do.
`numbering_level`	`Option<i64>`	`Default::default()`	Indentation level within that numbering definition (`w:numPr/w:ilvl`). Absent on most list styles. Word reads an absent level as 0.
`keep_next`	`Option<bool>`	`Default::default()`	Keep with next paragraph on same page.
`keep_lines`	`Option<bool>`	`Default::default()`	Keep all lines of paragraph on same page.
`page_break_before`	`Option<bool>`	`Default::default()`	Force page break before paragraph.
`widow_control`	`Option<bool>`	`Default::default()`	Prevent widow/orphan lines.
`suppress_auto_hyphens`	`Option<bool>`	`Default::default()`	Suppress automatic hyphenation.
`bidi`	`Option<bool>`	`Default::default()`	Right-to-left paragraph direction.
`shading_fill`	`Option<String>`	`Default::default()`	Background color hex value (from w:shd w:fill).
`shading_val`	`Option<String>`	`Default::default()`	Shading pattern value (from w:shd w:val).
`border_top`	`Option<String>`	`Default::default()`	Top border style (from w:pBdr/w:top w:val).
`border_bottom`	`Option<String>`	`Default::default()`	Bottom border style (from w:pBdr/w:bottom w:val).
`border_left`	`Option<String>`	`Default::default()`	Left border style (from w:pBdr/w:left w:val).
`border_right`	`Option<String>`	`Default::default()`	Right border style (from w:pBdr/w:right w:val).

PatternMatch

One detected PII span in the input text.

Field	Type	Default	Description
`start`	`usize`	—	Inclusive byte-offset start of the match in the source text.
`end`	`usize`	—	Exclusive byte-offset end of the match.
`category`	`PiiCategory`	—	Category the match belongs to.
`text`	`String`	—	Matched substring (owned copy — pattern engine returns owned data so the caller can free the original text if needed before replacement).

PcmAudio

The canonical PCM format that all transcription engines receive.

Field	Type	Default	Description
`samples`	`Vec<f32>`	—	Interleaved (or mono) f32 PCM samples in [-1.0, 1.0].
`sample_rate_hz`	`u32`	—	Always 16000 after resampling/normalization in this decoder.
`channels`	`u16`	—	Always 1 (mono) after our conversion.
`duration_ms`	`u64`	—	Duration of the decoded audio in milliseconds.

PdfAnnotation

A PDF annotation extracted from a document page.

Field	Type	Default	Description
`annotation_type`	`PdfAnnotationType`	—	The type of annotation.
`content`	`Option<String>`	`None`	Text content of the annotation (e.g., comment text, link URL).
`page_number`	`u32`	—	Page number where the annotation appears (1-indexed).
`bounding_box`	`Option<BoundingBox>`	`None`	Bounding box of the annotation on the page.
`author`	`Option<String>`	`/* serde(default) */`	Author/creator of the annotation (PDF `/T` entry).
`modified`	`Option<String>`	`/* serde(default) */`	Last modification date of the annotation (PDF `/M` entry), as a raw PDF date string (e.g. `"D:20240115120000Z"`).
`color`	`Option<String>`	`/* serde(default) */`	Annotation colour (PDF `/C` entry), normalised to a CSS-compatible `#rrggbb` hex string. Gray and CMYK colour spaces are converted to RGB.
`subject`	`Option<String>`	`/* serde(default) */`	Subject of the annotation (PDF `/Subj` entry).
`quad_points`	`Option<Vec<BoundingBox>>`	`/* serde(default) */`	Per-line bounding boxes derived from the annotation’s `/QuadPoints` entry. Present for text markup annotations (Highlight, Underline, StrikeOut, Squiggly), one box per marked line/run of text.
`marked_text`	`Option<String>`	`/* serde(default) */`	The document text covered by `Self::quad_points`, recovered from the page content underneath the marked-up region. Populated for Highlight, Underline, StrikeOut, and Squiggly annotations when the underlying text could be recovered.

PdfConfig

PDF-specific configuration.

Field	Type	Default	Description
`extract_images`	`bool`	`false`	Extract images from PDF
`extract_tables`	`bool`	`true`	Extract tables from PDF. When `true` (default), runs the native engine’s grid detector and, if it finds nothing, falls back to the heuristic text-layer reconstruction in `pdf::native::table::extract_tables_heuristic`. Set to `false` to skip both passes — `tables` will then be empty in the result.
`passwords`	`Option<Vec<String>>`	`None`	List of passwords to try when opening encrypted PDFs
`extract_metadata`	`bool`	`true`	Extract PDF metadata
`hierarchy`	`Option<HierarchyConfig>`	`None`	Hierarchy extraction configuration (None = hierarchy extraction disabled)
`extract_annotations`	`bool`	`false`	Extract PDF annotations (text notes, highlights, links, stamps). Default: false
`top_margin_fraction`	`Option<f32>`	`None`	Top margin fraction (0.0–1.0) of page height to exclude headers/running heads. Ignored when `ContentFilterConfig.include_headers` is `true`. Effective nonzero margins require per-page OCR so geometry can be filtered; document-capable OCR backends use their image-processing path in that case. Default: 0.0 (disabled; set explicitly to filter header content)
`bottom_margin_fraction`	`Option<f32>`	`None`	Bottom margin fraction (0.0–1.0) of page height to exclude footers/page numbers. Ignored when `ContentFilterConfig.include_footers` is `true`. Effective nonzero margins require per-page OCR so geometry can be filtered; document-capable OCR backends use their image-processing path in that case. Default: 0.0 (disabled; set explicitly to filter footer content)
`allow_single_column_tables`	`bool`	`false`	Allow single-column pseudo tables in extraction results. By default, tables with fewer than 2 columns (layout-guided) or 3 columns (heuristic) are rejected. When `true`, the minimum column count is relaxed to 1, allowing single-column structured data (glossaries, itemized lists) to be emitted as tables. Other quality filters (density, sparsity, prose detection) still apply.
`ocr_inline_images`	`bool`	`false`	Perform OCR on inline images extracted from PDF pages and attach the recognized text to each `ExtractedImage.ocr_result`. Uses the backend selected by `ExtractionConfig.ocr`, or the default OCR backend when no OCR configuration is supplied. Requires the `ocr` or `ocr-pipeline` feature. Per-image failures degrade gracefully (the image is returned without OCR text rather than failing the whole extraction). Default: `false`.
`extract_form_fields`	`bool`	`true`	Extract AcroForm and XFA form fields into `ExtractedDocument.form_fields`. When `true` (default), reads the document’s interactive form structure (field names, types, values, widget geometry). Cheap and strictly additive — non-form PDFs simply yield an empty list. Set to `false` to skip the form pass entirely.
`reading_order`	`bool`	`false`	Reorder extracted text by layout-detected reading order. When `true`, projects text spans onto layout-detected regions, performs column detection, and emits spans in natural reading order (important for multi-column academic PDFs). It also repairs 90/180/270-degree rotated text runs — sideways tables and captions — that otherwise read word-reversed and glued (GH#1358); see `crate::extractors::pdf::reading_order` for the rotation-handling details and its limits. Requires the `layout-detection` feature and a page for which layout detection actually produces hints: a page with no detected regions falls back to the original, unrepaired extraction order even with this enabled. Independent of `LayoutStrategy`, which only controls whether layout detection runs at all — enabling `Always` or `Auto` alone does not turn reordering on. Defaults to `false`.
`backend`	`PdfBackend`	`PdfBackend::Native`	Which engine parses and renders this PDF. Defaults to `PdfBackend::Native`. Selecting `PdfBackend::Pdfium` requires the `pdf-pdfium` feature and is rejected otherwise; the pdfium engine is also deliberately narrower in scope than `Native` – see `PdfBackend` and `extractors::pdf::pdfium_engine` for details.

Methods

default()

Signature:

pub fn default() -> PdfConfig

Example:

let result = PdfConfig::default();

Returns: PdfConfig

validate()

Validate PDF-specific extraction settings.

Errors:

Returns Validation when a configured page margin is non-finite or outside [0.0, 1.0], when hierarchy clustering requests fewer than one or more than seven clusters, or when an enabled option is unavailable in the current feature set.

Signature:

pub fn validate(&self) -> Result<(), Error>

Example:

instance.validate()?;

Returns: No return value.

Errors: Returns Err(Error).

PdfExtractionMetadata

Complete PDF extraction metadata including common and PDF-specific fields.

Combines common document fields (title, authors, dates) with PDF-specific metadata and optional page structure information.

Field	Type	Default	Description
`title`	`Option<String>`	`None`	Document title
`subject`	`Option<String>`	`None`	Document subject or description
`authors`	`Option<Vec<String>>`	`None`	Document authors (parsed from PDF Author field)
`keywords`	`Option<Vec<String>>`	`None`	Document keywords (parsed from PDF Keywords field)
`created_at`	`Option<String>`	`None`	Creation timestamp (ISO 8601 format)
`modified_at`	`Option<String>`	`None`	Last modification timestamp (ISO 8601 format)
`created_by`	`Option<String>`	`None`	Application or user that created the document
`pdf_specific`	`PdfMetadata`	—	PDF-specific metadata
`page_structure`	`Option<PageStructure>`	`None`	Page structure with boundaries and optional per-page metadata

PdfExtractor

PDF document extractor using xberg_native_pdf.

Methods

default()

Signature:

pub fn default() -> PdfExtractor

Example:

let result = PdfExtractor::default();

Returns: PdfExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

PdfFormField

A form field extracted from a PDF’s AcroForm or XFA structure.

Populated by the PDF extractor when PdfConfig::extract_form_fields is enabled and the document is a fillable form. Supports both AcroForm (standard) and XFA (XML Forms Architecture) layers. When both are present, AcroForm fields take priority (canonical fallback per PDF spec), and XFA-only fields are appended. The collection is empty for non-form PDFs and for non-PDF formats.

Field	Type	Default	Description
`name`	`String`	—	Partial field name (the leaf name within the field hierarchy).
`full_name`	`String`	—	Fully-qualified field name (dotted path from the form root).
`field_type`	`FormFieldType`	—	Classified field type.
`value`	`Option<String>`	`/* serde(default) */`	Current field value, if any.
`default_value`	`Option<String>`	`/* serde(default) */`	Default field value, if any.
`flags`	`u32`	`/* serde(default) */`	Raw field-flags bitmask (read-only, required, multiline, …).
`page`	`Option<u32>`	`/* serde(default) */`	1-indexed page the field’s widget appears on. Currently always `None` for AcroForm fields; page assignment is a deferred enhancement requiring spatial analysis of widget annotations per page.
`bbox`	`Option<BoundingBox>`	`/* serde(default) */`	Widget bounding box on its page, if known.
`max_length`	`Option<u32>`	`/* serde(default) */`	Maximum input length for text fields, if specified.
`tooltip`	`Option<String>`	`/* serde(default) */`	Tooltip / alternate field description, if present.

PdfMetadata

PDF-specific metadata.

Contains metadata fields specific to PDF documents that are not in the common Metadata structure. Common fields like title, authors, keywords, and dates are at the Metadata level.

Field	Type	Default	Description
`pdf_version`	`Option<String>`	`Default::default()`	PDF version (e.g., “1.7”, “2.0”)
`producer`	`Option<String>`	`Default::default()`	PDF producer (application that created the PDF)
`is_encrypted`	`Option<bool>`	`Default::default()`	Whether the PDF is encrypted/password-protected
`width`	`Option<i64>`	`Default::default()`	First page width in points (1/72 inch)
`height`	`Option<i64>`	`Default::default()`	First page height in points (1/72 inch)
`page_count`	`Option<u32>`	`Default::default()`	Total number of pages in the PDF document
`scanned_confidence`	`Option<f32>`	`Default::default()`	How strongly the document’s most scan-like page resembles a scan, in `[0.0, 1.0]`. `None` when the document could not be inspected. A full-page raster with no visible text scores at least `0.85`; a born-digital slide with a full-bleed background image scores `0.50`.
`scanned_pages`	`Option<Vec<u32>>`	`vec![]`	Pages that look like scans (1-indexed), using the default confidence threshold. `None` when the document could not be inspected; empty when no page qualifies.
`fabricated_text_pages`	`Option<Vec<u32>>`	`vec![]`	Pages whose text was dominated by fabricated character mappings (1-indexed): `MappingProvenance::Fallback`, a font whose glyph-to-Unicode mapping resolved to a value the extractor chose rather than read from the file (issue #1254). This is a fact about how the text was derived, independent of `scanned_pages`’s raster-based scan detection, and independent of whether the resulting text happens to look structurally like prose (issue #1667: a broken mapping that lands on ordinary letters and punctuation passes every character-shape check but is still fabricated). `None` when `OcrQualityThresholds::enable_provenance_ocr_routing` is `false`; empty when no page qualifies.
`implausible_text_pages`	`Option<Vec<u32>>`	`vec![]`	Pages whose native text layer reads as no real detectable language (1-indexed): a `/ToUnicode` CMap (or other mapping tier) that resolves every glyph to a character, consistently the WRONG one (e.g. a ROT-shifted mapping), so the page is structurally indistinguishable from real prose to `fabricated_text_pages`’s provenance check and to every character-shape heuristic (issue #1696; issue #1667’s `quality_score: 1.0` with no warning on such a page is the same underlying gap). This is a content-plausibility fact, independent of `fabricated_text_pages` and of `scanned_pages`. `None` when `OcrQualityThresholds::enable_plausibility_ocr_routing` is `false`; empty when no page qualifies.
`layout_gated_pages`	`Option<Vec<u32>>`	`vec![]`	Pages the `auto` layout strategy skipped (1-indexed). `None` unless layout detection ran with `LayoutStrategy::Auto`; empty when the gate selected every page.
`layout_gate_reasons`	`Option<Vec<String>>`	`vec![]`	Why the `auto` layout gate selected or skipped each page. Index `i` is page `i + 1`. Snake_case values such as `multi_column`, `table_grid`, or `plain_text` (the skip reason). `None` unless layout detection ran with `LayoutStrategy::Auto`.

PdfRenderSession

An open PDF document that can render multiple pages without reparsing the file.

This Rust-only session keeps the native PDF engine private while exposing the efficient open-once rendering path. Use render_pdf_page_to_png for a single page and this type when rendering several pages from the same document.

PixelDimensions

Image preprocessing metadata.

Tracks the transformations applied to an image during OCR preprocessing, including DPI normalization, resizing, and resampling.

Field	Type	Default	Description
`width`	`usize`	—	Width in pixels.
`height`	`usize`	—	Height in pixels.

PlainTextExtractor

Plain text extractor.

Extracts content from plain text files (.txt).

Methods

default()

Signature:

pub fn default() -> PlainTextExtractor

Example:

let result = PlainTextExtractor::default();

Returns: PlainTextExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Only text/plain.

This extractor used to also claim text/asciidoc, text/x-asciidoc, text/vtt, text/troff, text/x-mdoc, text/x-pod and text/x-dokuwiki while doing nothing beyond a BOM strip and a blank-line paragraph split, so every one of those formats silently produced structureless garbage that looked like a successful extraction (#228).

AsciiDoc and WebVTT now have real extractors (AsciiDocExtractor, WebVttExtractor). troff, mdoc, POD and DokuWiki were dropped from the format catalogue entirely rather than advertised with no implementation behind them.

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Plugin

Base trait that all plugins must implement.

This trait provides common functionality for plugin lifecycle management, identification, and metadata.

Thread Safety

All plugins must be Send + Sync to support concurrent usage across threads.

Methods

name()

Returns the unique name/identifier for this plugin.

The name should be:

Unique across all plugins
Lowercase with hyphens (e.g., “my-custom-plugin”)
URL-safe characters only

Signature:

pub fn name(&self) -> String

Example:

fn name(&self) -> &str {
    "pdf-extractor"
}

Returns: String

version()

Returns the semantic version of this plugin.

Should follow semver format: MAJOR.MINOR.PATCH

Defaults to the xberg crate version.

Signature:

pub fn version(&self) -> String

Example:

fn version(&self) -> String {
    "1.2.3".to_string()
}

Defaults to the xberg crate version.

Returns: String

initialize()

Initialize the plugin.

Called once when the plugin is registered. Use this to:

Load configuration
Initialize resources (connections, caches, etc.)
Validate dependencies

Thread Safety

This method takes &self instead of &mut self to work with Arc<dyn Plugin>. Plugins needing mutable state during initialization should use interior mutability patterns (Mutex, RwLock, OnceCell, etc.).

Errors:

Should return an error if initialization fails. The plugin will not be registered if this method returns an error.

Defaults to a no-op for stateless plugins.

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

fn initialize(&self) -> Result<()> {
    // Load configuration using interior mutability
    let mut config = self.config.lock().unwrap();
    *config = Some("loaded".to_string());

    // Perform any initialization work
    println!("Plugin initialized successfully");

    Ok(())
}

Defaults to a no-op for stateless plugins.

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Shutdown the plugin.

Called when the plugin is being unregistered or the application is shutting down. Use this to:

Close connections
Flush caches
Release resources

Thread Safety

This method takes &self instead of &mut self to work with Arc<dyn Plugin>. Plugins needing mutable state during shutdown should use interior mutability patterns (Mutex, RwLock, etc.).

Errors:

Errors during shutdown are logged but don’t prevent the shutdown process.

Defaults to a no-op for stateless plugins.

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

fn shutdown(&self) -> Result<()> {
    // Flush caches using interior mutability
    let mut cache = self.cache.lock().unwrap();
    if let Some(data) = cache.take() {
        // Persist cache to disk
    }

    Ok(())
}

Defaults to a no-op for stateless plugins.

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Optional plugin description for debugging and logging.

Defaults to empty string if not overridden.

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Optional plugin author information.

Defaults to empty string if not overridden.

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

PluginHealthStatus

Plugin health status information.

Contains diagnostic information about registered plugins for each type.

Field	Type	Default	Description
`ocr_backends_count`	`usize`	—	Number of registered OCR backends
`ocr_backends`	`Vec<String>`	—	Names of registered OCR backends
`extractors_count`	`usize`	—	Number of registered document extractors
`extractors`	`Vec<String>`	—	Names of registered document extractors
`post_processors_count`	`usize`	—	Number of registered post-processors
`post_processors`	`Vec<String>`	—	Names of registered post-processors
`validators_count`	`usize`	—	Number of registered validators
`validators`	`Vec<String>`	—	Names of registered validators

Position

Horizontal or vertical position.

Field	Type	Default	Description
`relative_from`	`String`	—	Reference object for this position: `"page"`, `"margin"`, `"column"`, `"paragraph"`, or `"character"`.
`offset`	`Option<i64>`	`Default::default()`	Offset from the reference object in EMUs.

PostProcessor

Trait for post-processor plugins.

Post-processors transform or enrich extraction results after the initial extraction is complete. They can:

Clean and normalize text
Add metadata (language, keywords, entities)
Split content into chunks
Score quality
Apply custom transformations

Processing Order

Post-processors are executed in stage order:

Early - Language detection, entity extraction
Middle - Keyword extraction, token reduction
Late - Custom hooks, final validation

Within each stage, processors are executed in registration order.

Error Handling

Post-processor errors are non-fatal by default - they’re captured in metadata and execution continues. To make errors fatal, return an error from process().

Thread Safety

Post-processors must be thread-safe (Send + Sync).

Methods

process()

Process an extraction result.

Transform or enrich the extraction result. Can modify:

content - The extracted text
metadata - Add or update metadata fields
tables - Modify or enhance table data

Returns:

Ok(()) if processing succeeded, Err(...) for fatal failures.

Errors:

Return errors for fatal processing failures. Non-fatal errors should be captured in metadata directly on the result.

Performance

This signature avoids unnecessary cloning of large extraction results by taking a mutable reference instead of ownership. Processors modify the result in place.

Example - Language Detection

Example - Text Cleaning

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	Mutable reference to the extraction result to process
`config`	`&ExtractionConfig`	Yes	Extraction configuration

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Get the processing stage for this post-processor.

Determines when this processor runs in the pipeline.

Returns:

The ProcessingStage (Early, Middle, or Late).

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

fn processing_stage(&self) -> ProcessingStage {
    ProcessingStage::Early  // Run before other processors
}

Returns: ProcessingStage

should_process()

Optional: Check if this processor should run for a given result.

Allows conditional processing based on MIME type, metadata, or content. Defaults to true (always run).

Returns:

true if the processor should run, false to skip.

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

/// Only process PDF documents
fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool {
    result.mime_type == "application/pdf"
}

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: bool

estimated_duration_ms()

Optional: Estimate processing time in milliseconds.

Used for logging and debugging. Defaults to 0 (unknown).

Returns:

Estimated processing time in milliseconds.

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

priority()

Execution priority within the processing stage.

Higher values run first within the same ProcessingStage. Defaults to 50. Use 0-49 for fallback processors, 50 for normal processors, and 51-255 for high-priority processors that should run early in their stage.

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

PostProcessorConfig

Post-processor configuration.

Field	Type	Default	Description
`enabled`	`bool`	`true`	Enable post-processors
`enabled_processors`	`Option<Vec<String>>`	`None`	Whitelist of processor names to run (None = all enabled)
`disabled_processors`	`Option<Vec<String>>`	`None`	Blacklist of processor names to skip (None = none disabled)
`enabled_set`	`Option<Vec<String>>`	`None`	Pre-computed AHashSet for O(1) enabled processor lookup
`disabled_set`	`Option<Vec<String>>`	`None`	Pre-computed AHashSet for O(1) disabled processor lookup

Methods

default()

Signature:

pub fn default() -> PostProcessorConfig

Example:

let result = PostProcessorConfig::default();

Returns: PostProcessorConfig

PostProcessorRegistry

Registry for post-processor plugins.

Manages post-processors organized by processing stage.

Methods

default()

Signature:

pub fn default() -> PostProcessorRegistry

Example:

let result = PostProcessorRegistry::default();

Returns: PostProcessorRegistry

PpDocLayoutV3Model

PP-DocLayout-V3 layout detection model.

Uses PaddleDetection DETR architecture with 800 × 800 input. Preprocessing squash-resizes to 800 × 800 (/255 normalisation only; no ImageNet mean/std), and passes im_shape/scale_factor so the model itself unscales output boxes to original-image pixel coordinates.

Methods

detect()

Signature:

pub fn detect(&mut self, img: &RgbImage) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect(&todo!())?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_with_threshold()

Signature:

pub fn detect_with_threshold(&mut self, img: &RgbImage, threshold: f32) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect_with_threshold(&todo!(), 0.5)?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img
`threshold`	`f32`	Yes	The threshold

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_batch()

Signature:

pub fn detect_batch(&mut self, images: &Vec<RgbImage>, threshold: Option<f32>) -> Result<Vec<Vec<LayoutDetection>>, LayoutError>

Example:

let result = instance.detect_batch(&todo!(), Some(0.5))?;

Parameters:

Name	Type	Required	Description
`images`	`&Vec<RgbImage>`	Yes	The images
`threshold`	`Option<f32>`	No	The threshold

Returns: Vec<Vec<LayoutDetection>>

Errors: Returns Err(LayoutError).

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

PptExtractionResult

Result of PPT text extraction.

Field	Type	Default	Description
`text`	`String`	—	Full document text: every slide’s text joined by double newlines, kept for diagnostics/plain-text consumers. Slide structure (numbers, per-slide boundaries) must come from `slides`, not from re-splitting this string (see #1418 – a slide’s own text can itself contain an internal “\n\n”, which makes re-splitting on it ambiguous).
`slides`	`Vec<PptSlideText>`	—	Per-slide text, in persist order. `number` is the slide’s 1-based position among `RT_SLIDE` containers as they occur in the “PowerPoint Document” stream – the same order `slide_count` counts. A slide with no text atoms still gets an entry (with an empty `text`), so slide numbers stay contiguous with the real deck.
`slide_count`	`usize`	—	Number of slides found.
`metadata`	`PptMetadata`	—	Document metadata.
`speaker_notes`	`Vec<String>`	—	Speaker notes text per slide (if available).
`images`	`Vec<ExtractedImage>`	—	Pictures recovered from the OLE `Pictures` stream (raw `OfficeArtBlip` payloads). Empty when the deck has no `Pictures` stream, the stream is empty, or image extraction was not requested. `page_number` is the slide whose drawing displays the picture, resolved through `pib` -> `msofbtBSE.foDelay`; it stays `None` for a blip no live shape references (#1620).
`processing_warnings`	`Vec<ProcessingWarning>`	—	Non-fatal degradations encountered while extracting (see `core::diagnostics`). Empty when extraction was complete.

PptExtractor

Native PPT extractor using OLE/CFB parsing.

This extractor handles PowerPoint 97-2003 binary (.ppt) files without requiring LibreOffice, providing ~50x faster extraction.

Methods

default()

Signature:

pub fn default() -> PptExtractor

Example:

let result = PptExtractor::default();

Returns: PptExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

PptMetadata

Metadata extracted from PPT files.

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	Presentation title from the OLE summary information.
`subject`	`Option<String>`	`Default::default()`	Presentation subject from the OLE summary information.
`author`	`Option<String>`	`Default::default()`	Original author from the OLE summary information.
`last_author`	`Option<String>`	`Default::default()`	Most recent editor from the OLE summary information.

PptSlideText

One slide’s text, numbered by its position in the deck’s own persist order rather than by the position of a text block in a joined string.

Field	Type	Default	Description
`number`	`u32`	—	1-based slide number, as encountered in persist order.
`text`	`String`	—	The slide’s text (its atoms joined by `\n`). Empty for a slide with no text.
`title`	`Option<String>`	`None`	The slide’s title, read from the outline collection’s own `TextHeaderAtom` (`Title` / `CenterTitle`) rather than guessed from `text`’s first line (xberg-io/xberg#1635). `None` when the file states no outline title for this slide – a deck whose title is drawn on the canvas and never entered in the outline view still has one, just not here; the consumer’s first-line fallback covers that case.
`notes`	`Option<String>`	`None`	This slide’s speaker notes, resolved via `NotesAtom.slideIdRef` against the live `SlideListWithText` (xberg-io/xberg#1640) – not by position among the deck’s non-empty notes pages, which misattributes a note as soon as one slide in between has none. `None` when the slide has no notes, or the file cannot be read that way.

PptxAppProperties

Application properties from docProps/app.xml for PPTX

Contains PowerPoint-specific document metadata.

Field	Type	Default	Description
`application`	`Option<String>`	`Default::default()`	Application name (e.g., “Microsoft Office PowerPoint”)
`app_version`	`Option<String>`	`Default::default()`	Application version
`total_time`	`Option<i32>`	`Default::default()`	Total editing time in minutes
`company`	`Option<String>`	`Default::default()`	Company name
`doc_security`	`Option<i32>`	`Default::default()`	Document security level
`scale_crop`	`Option<bool>`	`Default::default()`	Scale crop flag
`links_up_to_date`	`Option<bool>`	`Default::default()`	Links up to date flag
`shared_doc`	`Option<bool>`	`Default::default()`	Shared document flag
`hyperlinks_changed`	`Option<bool>`	`Default::default()`	Hyperlinks changed flag
`slides`	`Option<i32>`	`Default::default()`	Number of slides
`notes`	`Option<i32>`	`Default::default()`	Number of notes
`hidden_slides`	`Option<i32>`	`Default::default()`	Number of hidden slides
`multimedia_clips`	`Option<i32>`	`Default::default()`	Number of multimedia clips
`presentation_format`	`Option<String>`	`Default::default()`	Presentation format (e.g., “Widescreen”, “Standard”)
`slide_titles`	`Vec<String>`	`vec![]`	Slide titles

PptxExtractionOptions

Options for PPTX content extraction.

Field	Type	Default	Description
`extract_images`	`bool`	`true`	Whether to extract embedded images.
`page_config`	`Option<PageConfig>`	`None`	Optional page configuration for boundary tracking.
`plain`	`bool`	`false`	Whether to output plain text (no markdown).
`include_structure`	`bool`	`false`	Whether to build the `DocumentStructure` tree.
`inject_placeholders`	`bool`	`true`	Whether to emit `!alt` references in markdown output.
`security_limits`	`SecurityLimits`	—	Security limits applied at container open: entry count (`max_files_in_archive`), aggregate uncompressed size (`max_archive_size`), and compression ratio (`max_compression_ratio`), from `ExtractionConfig.security_limits`. Defaults to `SecurityLimits::default()` when unset.
`max_pages`	`Option<usize>`	`Default::default()`	Maximum number of slides the presentation may contain, from `ExtractionConfig.security_limits.max_pages` (#1451). `None` (the default) means unlimited.

Methods

default()

Signature:

pub fn default() -> PptxExtractionOptions

Example:

let result = PptxExtractionOptions::default();

Returns: PptxExtractionOptions

PptxExtractionResult

PowerPoint (PPTX) extraction result.

Contains extracted slide content, metadata, and embedded images/tables.

Field	Type	Default	Description
`content`	`String`	—	Extracted text content from all slides
`metadata`	`PptxMetadata`	—	Presentation metadata
`slide_count`	`usize`	—	Total number of slides
`image_count`	`usize`	—	Total number of embedded images
`table_count`	`usize`	—	Total number of tables
`images`	`Vec<ExtractedImage>`	—	Extracted images from the presentation
`page_structure`	`Option<PageStructure>`	`None`	Slide structure with boundaries (when page tracking is enabled)
`page_contents`	`Option<Vec<PageContent>>`	`None`	Per-slide content (when page tracking is enabled)
`document`	`Option<DocumentStructure>`	`None`	Structured document representation
`hyperlinks`	`Vec<PresentationHyperlink>`	`/* serde(default) */`	Hyperlinks discovered in slides.
`office_metadata`	`HashMap<String, String>`	`/* serde(default) */`	Office metadata extracted from docProps/core.xml and docProps/app.xml. Contains keys like “title”, “author”, “created_by”, “subject”, “keywords”, “modified_by”, “created_at”, “modified_at”, etc.
`revisions`	`Option<Vec<DocumentRevision>>`	`/* serde(default) */`	Slide comments as revisions. Each `<p:cm>` element in `ppt/comments/comment{N}.xml` becomes a `DocumentRevision { kind: Comment }` with author (resolved from `ppt/commentAuthors.xml`), ISO-8601 timestamp, and `RevisionAnchor::Slide { index }`. `None` when no comment XML parts exist.

PptxExtractor

PowerPoint presentation extractor.

Supports: .pptx, .pptm, .ppsx

Methods

default()

Signature:

pub fn default() -> PptxExtractor

Example:

let result = PptxExtractor::default();

Returns: PptxExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

PptxMetadata

PowerPoint presentation metadata.

Extracted from PPTX files containing slide counts and presentation details.

Field	Type	Default	Description
`slide_count`	`u32`	—	Total number of slides in the presentation
`slide_names`	`Vec<String>`	`vec![]`	Names of slides (if available)
`image_count`	`Option<u32>`	`Default::default()`	Number of embedded images
`table_count`	`Option<u32>`	`Default::default()`	Number of tables

PreprocessingOptions

HTML preprocessing options for document cleanup before conversion.

Field	Type	Default	Description
`enabled`	`bool`	`true`	Enable HTML preprocessing globally
`preset`	`PreprocessingPreset`	`PreprocessingPreset::Standard`	Preprocessing preset level (Minimal, Standard, Aggressive)
`remove_navigation`	`bool`	`true`	Remove navigation elements (nav, breadcrumbs, menus, sidebars)
`remove_forms`	`bool`	`true`	Remove form elements (forms, inputs, buttons, etc.)

PresentationHyperlink

A hyperlink discovered in a presentation slide.

Field	Type	Default	Description
`url`	`String`	—	Link destination.
`label`	`Option<String>`	`Default::default()`	Optional visible label.

Preset

A curated structured-extraction preset loaded from the embedded library.

Each preset is a JSON file under src/presets/library/<id>/v1.json that validates against the meta-schema in src/presets/preset.schema.json.

Downstream catalog consumers can inject presets via extend_from_dir. The embedded OSS library ships only the generic_document toy preset.

Field	Type	Default	Description
`id`	`String`	—	Stable, URL-safe preset identifier (lowercase snake_case).
`version`	`String`	—	Monotonic version string (e.g. `v1`).
`schema_name`	`String`	—	Human-readable schema name forwarded to the LLM as the response/tool name.
`description`	`String`	—	One-line preset description shown in the registry UI.
`category`	`PresetCategory`	—	Top-level category for grouping in the playground.
`tags`	`Vec<String>`	`/* serde(default) */`	Free-form tags used for search/filtering. May be empty.
`schema`	`serde_json::Value`	—	JSON Schema (Draft 2020-12) describing the structured output shape.
`system_prompt`	`String`	—	Instruction primer sent to the model.
`context_template`	`Option<String>`	`/* serde(default) */`	Optional mustache-style template merged with caller-supplied context.
`merge_mode`	`MergeMode`	—	Strategy for merging per-batch outputs across paginated calls.
`preferred_call_mode`	`CallMode`	—	Default call mode suggested for this preset; heuristics may override.
`emit_citations`	`bool`	—	When true, the prompt asks the model to wrap each field as `{value, page, bbox, confidence}` for downstream citation overlays.
`sample`	`Option<PresetSample>`	`/* serde(default) */`	Optional bundled sample (input file + reference output) for preview.
`fingerprint`	`String`	`/* serde(default) */`	Stable sha256 fingerprint of the canonical preset file contents. Populated at registry load — not present in the on-disk JSON files. Used as a cache-invalidation token by the worker pipeline.

PresetSample

Pointer to a sample input + its reference output bundled with the preset.

Field	Type	Default	Description
`input_path`	`String`	—	Path to the sample input file, relative to the preset directory.
`output_path`	`String`	—	Path to the reference structured output, relative to the preset directory.

PresetSummary

Lightweight projection of Preset used by the registry list endpoint (omits the full schema and prompt to keep the payload small).

Field	Type	Default	Description
`id`	`String`	—	Preset identifier matching `Preset::id`.
`version`	`String`	—	Preset version matching `Preset::version`.
`schema_name`	`String`	—	Schema name matching `Preset::schema_name`.
`description`	`String`	—	One-line preset description.
`category`	`PresetCategory`	—	Top-level category.
`tags`	`Vec<String>`	—	Free-form tags.
`preferred_call_mode`	`CallMode`	—	Default call mode.
`emit_citations`	`bool`	—	Whether the preset prompts the model for citations.
`fingerprint`	`String`	—	Stable fingerprint matching `Preset::fingerprint`.

ProcessingWarning

A non-fatal warning from a processing pipeline stage.

Captures errors from optional features that don’t prevent extraction but may indicate degraded or incomplete results. Inspect these independently from ExtractedDocument::quality_score, which assesses retained text only.

Field	Type	Default	Description
`source`	`String`	—	The pipeline stage or feature that produced this warning (e.g., “embedding”, “chunking”, “language_detection”, “output_format”).
`message`	`String`	—	Human-readable description of what went wrong.

PropertyChange

A single run-level or style-level property change.

Used for revisions that change formatting rather than text content. from and to store normalized property values when the source format exposes them; either side may be absent when the format only records one side of the change.

Field	Type	Default	Description
`name`	`String`	—	Property name, such as `"bold"`, `"italic"`, `"font_size"`, or `"font_color"`.
`from`	`Option<String>`	`None`	Value before the change, when available.
`to`	`Option<String>`	`None`	Value after the change, when available.

ProxyConfig

Proxy configuration for HTTP requests.

Field	Type	Default	Description
`url`	`String`	—	Proxy URL (e.g. “http://proxy:8080”).
`username`	`Option<String>`	`Default::default()`	Optional username for proxy authentication.
`password`	`Option<String>`	`Default::default()`	Optional password for proxy authentication.

PstExtractor

PST file extractor.

Supports: .pst (Microsoft Outlook Personal Folders)

Methods

default()

Signature:

pub fn default() -> PstExtractor

Example:

let result = PstExtractor::default();

Returns: PstExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_sync()

Signature:

pub fn extract_sync(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_sync(b"data", "value", &todo!())?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

PstMetadata

Outlook PST archive metadata.

Field	Type	Default	Description
`message_count`	`usize`	—	Total number of email messages found in the PST archive.

QrBoundingBox

Pixel-space bounding box of a QR code inside its source image.

Field	Type	Default	Description
`x`	`u32`	—	Horizontal pixel offset of the bounding box top-left corner.
`y`	`u32`	—	Vertical pixel offset of the bounding box top-left corner.
`width`	`u32`	—	Width of the bounding box in pixels.
`height`	`u32`	—	Height of the bounding box in pixels.

QrCode

One QR code decoded from an extracted image.

Field	Type	Default	Description
`payload`	`String`	—	Decoded payload (text, URL, vCard string, …).
`confidence`	`Option<f32>`	`None`	Detector-reported confidence in `[0.0, 1.0]`. `None` when the decoder does not expose confidence (the default `rqrr` backend always reports `Some` because successful decode implies high confidence).
`bbox`	`Option<QrBoundingBox>`	`None`	Bounding box of the QR code inside the source image, in pixel coordinates (`x`, `y` of the top-left corner; `width`, `height` of the rectangle). `None` if the decoder did not report a bounding box.

QrCodeProcessor

Post-processor that runs rqrr over each image, writes the decoded QR payloads into qr_codes, and surfaces them in the document text (and, for URL payloads, in ExtractedDocument::uris) so downstream enrichment sees them.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

QualityProcessor

Post-processor that calculates a quality score.

This processor:

Runs in the Early processing stage
Calculates quality score when config.enable_quality_processing is true
Stores the text cleanliness/readability score in ExtractedDocument::quality_score

The score describes retained text, not extraction completeness or recall. Callers must inspect ExtractedDocument::processing_warnings separately for known omissions and degraded processing.

When OCR ran and recognized enough words to trust the result, the score is capped at the word-count-weighted mean OCR recognition confidence (issue #1669): the text-shape heuristic in text::quality::calculate_quality_score reads only the retained text itself, so a page the OCR engine recognized at 81% confidence can still look shape-clean and score 1.0. That mean is folded through ConfidenceSignals::ocr_confidence_from_elements / ocr_confidence_from_pages – the same fns ExtractionConfidence::ocr_aggregate uses, so this cap and that field never drift onto different weightings (issue #1694). The evidence floor before the cap trusts the mean is independently calibrated and stricter than ocr_aggregate’s own; see MIN_OCR_WORDS_FOR_CONFIDENCE_FLOOR. That confidence is a fact about the trustworthiness of the SAME retained text quality_score already claims to describe, not a completeness signal, so folding it in here does not reopen the completeness/quality boundary quality_score_does_not_hide_completeness_warning establishes below.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

estimated_duration_ms()

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

RakeParams

RAKE-specific parameters.

Field	Type	Default	Description
`min_word_length`	`usize`	`1`	Minimum word length to consider (default: 1).
`max_words_per_phrase`	`usize`	`3`	Maximum words in a keyword phrase (default: 3).

Methods

default()

Signature:

pub fn default() -> RakeParams

Example:

let result = RakeParams::default();

Returns: RakeParams

RecModelPaths

Paths to a recognition model and its character dictionary.

Field	Type	Default	Description
`rec_model`	`PathBuf`	—	Exact path to the recognition ONNX model in the Hugging Face snapshot.
`dict_file`	`PathBuf`	—	Path to the character dictionary file.

RecognizedTable

Pre-computed table markdown for a table detection region.

Produced by the TATR-based table structure recognizer and surfaced as part of layout-aware OCR results. The struct lives here (under layout-types, pure-Rust) so that consumers who do not enable layout-detection (ORT) can still reference the type in their own code.

Field	Type	Default	Description
`detection_bbox`	`BBox`	—	Detection bbox that this table corresponds to (for matching).
`cells`	`Vec<Vec<String>>`	—	Table cells as a 2D vector (rows × columns).
`markdown`	`String`	—	Rendered markdown table.

Record

A single HWP binary record decoded from a stream.

Each record starts with a 32-bit packed header containing the tag ID, nesting level, and payload size; followed by the raw payload bytes.

Field	Type	Default	Description
`tag_id`	`u16`	—	Tag identifier (10-bit value from the packed record header).
`level`	`u16`	—	Nesting level (10-bit value from the packed record header). Body-text records nested inside a control (table cells, embedded objects) carry a level higher than their containing paragraph; used to find where a table/cell ends (#105).
`data`	`Vec<u8>`	—	Raw payload bytes for this record.

RedactionConfig

Since: v1.0

Configuration for the redaction post-processor.

Field	Type	Default	Description
`categories`	`Vec<PiiCategory>`	`vec![]`	Categories to redact. Empty means “every category supported by the engine.”
`strategy`	`RedactionStrategy`	`RedactionStrategy::Mask`	Strategy applied to every match.
`ner`	`Option<NerConfig>`	`None`	Optional NER backend — required to redact PERSON / ORGANIZATION / LOCATION categories (the pure-Rust pattern engine only covers regex-detectable PII).
`preserve_offsets`	`bool`	`true`	When `true`, chunk byte ranges are kept consistent with the rewritten content by adjusting `byte_start` / `byte_end` after replacement. When `false`, chunk byte ranges still refer to the original content offsets — useful when downstream consumers want to map findings back to the original document.
`custom_terms`	`Vec<RedactionTerm>`	`vec![]`	Arbitrary user-supplied literal terms to redact. Each term is treated as a regex hit against the document, surfacing as `PiiCategory::Custom(label)` in `RedactionFinding` where `label` is the per-term label (defaulting to the literal value itself). Case-insensitive by default; set `RedactionTerm::case_sensitive` for exact match. Use this when you need to redact tenant-specific tokens (employee IDs, project codes, internal product names) without writing a custom plugin.
`custom_patterns`	`Vec<RedactionPattern>`	`vec![]`	Arbitrary user-supplied regex patterns to redact. Same surfacing semantics as `custom_terms`: each hit becomes a `PiiCategory::Custom(label)` finding. Patterns are validated at config-construction time via `RedactionConfig::validate`.
`findings`	`Vec<ExternalRedactionFinding>`	`vec![]`	Findings supplied inline by an external inspection engine.
`min_score`	`Option<f32>`	`None`	Minimum accepted confidence for scored external findings. Findings without a score remain eligible.
`findings_path`	`Option<PathBuf>`	`None`	JSON array or JSON Lines file containing external findings.
`findings_offset_encoding`	`RedactionOffsetEncoding`	`RedactionOffsetEncoding::UnicodeCodePoints`	Unit used by configured findings that derive their text from offsets.

Methods

default()

Signature:

pub fn default() -> RedactionConfig

Example:

let result = RedactionConfig::default();

Returns: RedactionConfig

validate()

Validate user-supplied terms and patterns at config-construction time.

Compiles every RedactionPattern::pattern (with the case-insensitive inline flag where applicable) and returns the first compilation error so the caller can reject the config before the redaction pipeline runs. Pure terms (regex-escaped) cannot fail to compile, but the function still rejects empty values to avoid degenerate zero-length matches.

Signature:

pub fn validate(&self) -> Result<(), Error>

Example:

instance.validate()?;

Returns: No return value.

Errors: Returns Err(Error).

RedactionFinding

One redaction event: which span was rewritten, why, and with what.

Field	Type	Default	Description
`start`	`u32`	—	Byte-offset start in the original (pre-redaction) `ExtractedDocument::content`.
`end`	`u32`	—	Byte-offset end (exclusive) in the original `ExtractedDocument::content`.
`category`	`PiiCategory`	—	PII category that fired this redaction.
`strategy`	`RedactionStrategy`	—	Strategy applied to this finding (mask, hash, token-replace, drop).
`replacement_token`	`String`	—	String that replaced the original mention. Always present; for `Drop` the replacement is the empty string.

RedactionPattern

One user-supplied regex pattern to redact.

The pattern is compiled with the Rust regex crate (no look-around). Case sensitivity is encoded in the pattern via the (?i) inline flag when Self::case_sensitive is false.

Field	Type	Default	Description
`label`	`String`	—	Custom category label surfaced in `RedactionFinding::category`.
`pattern`	`String`	—	Regex pattern (Rust `regex` crate dialect — no look-around).
`case_sensitive`	`bool`	`false`	When `true`, match case-sensitively; otherwise prepend `(?i)` to the regex.

Methods

labeled()

Build a pattern with the given label (case-insensitive by default).

Signature:

pub fn labeled(label: String, pattern: String) -> RedactionPattern

Example:

let result = RedactionPattern::labeled("value", "value");

Parameters:

Name	Type	Required	Description
`label`	`String`	Yes	The label
`pattern`	`String`	Yes	The pattern

Returns: RedactionPattern

RedactionProcessor

Redaction post-processor.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

RedactionReport

Audit report describing what the redaction processor found and how it replaced it.

The redactor returns this alongside the rewritten content so compliance, replay, and audit-log consumers can see exactly what fired. Offsets are relative to the original pre-redaction content and are intended for audit reconstruction only — the original bytes are dropped at the end of the pipeline.

Field	Type	Default	Description
`findings`	`Vec<RedactionFinding>`	—	Individual redaction findings in original-source byte order.
`total_redacted`	`u32`	—	Total number of redactions applied across the document.

RedactionTerm

One user-supplied literal term to redact.

Matched as a regex-escaped substring (so callers do not need to escape metacharacters themselves). Case-insensitive by default — set Self::case_sensitive to true for exact byte-match semantics.

Field	Type	Default	Description
`label`	`String`	—	Custom category label surfaced in `RedactionFinding::category`.
`value`	`String`	—	Literal value to match. Regex metacharacters are escaped automatically.
`case_sensitive`	`bool`	`false`	When `true`, match the value as-is; otherwise match ASCII-case-insensitively.

Methods

literal()

Build a term whose label is the literal value itself (case-insensitive).

Signature:

pub fn literal(value: String) -> RedactionTerm

Example:

let result = RedactionTerm::literal("value");

Parameters:

Name	Type	Required	Description
`value`	`String`	Yes	The value

Returns: RedactionTerm

labeled()

Build a term with a custom label.

Signature:

pub fn labeled(label: String, value: String) -> RedactionTerm

Example:

let result = RedactionTerm::labeled("value", "value");

Parameters:

Name	Type	Required	Description
`label`	`String`	Yes	The label
`value`	`String`	Yes	The value

Returns: RedactionTerm

Registry

Sorted map of preset id → Preset.

Methods

load_embedded()

Build the registry from preset files embedded at compile time under src/presets/library/. Validates every file against the meta-schema.

Signature:

pub fn load_embedded() -> Result<Registry, LoadError>

Example:

let result = Registry::load_embedded()?;

Returns: Registry

Errors: Returns Err(LoadError).

global()

Return the global registry, loading it on first access.

Panics:

Panics if any embedded preset is malformed. The build-time validation test ensures this cannot happen for the embedded presets; a panic here indicates a build artifact problem, not a runtime error.

Signature:

pub fn global() -> Registry

Example:

let result = Registry::global();

Returns: Registry

get()

Look up a preset by its identifier.

Signature:

pub fn get(&self, id: &str) -> Option<Preset>

Example:

let result = instance.get("value");

Parameters:

Name	Type	Required	Description
`id`	`&str`	Yes	The id

Returns: Option<Preset>

summaries()

Materialize a PresetSummary list for the public registry endpoint.

Signature:

pub fn summaries(&self) -> Vec<PresetSummary>

Example:

let result = instance.summaries();

Returns: Vec<PresetSummary>

len()

Number of presets currently loaded.

Signature:

pub fn len(&self) -> usize

Example:

let result = instance.len();

Returns: usize

is_empty()

Whether the registry contains zero presets.

Signature:

pub fn is_empty(&self) -> bool

Example:

let result = instance.is_empty();

Returns: bool

sample_bytes()

Read raw sample bytes for <preset_id> from library/<id>/samples/<name>. Returns None when the file is absent.

Signature:

pub fn sample_bytes(&self, preset_id: &str, name: &str) -> Option<Vec<u8>>

Example:

let result = instance.sample_bytes("value", "value");

Parameters:

Name	Type	Required	Description
`preset_id`	`&str`	Yes	The preset id
`name`	`&str`	Yes	The name

Returns: Option<Vec<u8>>

extend_from_dir()

Load additional preset files from a runtime directory and insert them into this registry.

Reads every *.json file directly under dir (non-recursive), validates each against the meta-schema, and inserts it. Files that fail validation are rejected — the error is returned immediately and the registry is left in a partially-updated state. Existing entries with the same id are overwritten.

Returns the number of presets successfully loaded from dir.

Use case

This is the injection point for downstream catalogs that add curated presets on top of the single embedded OSS preset.

Signature:

pub fn extend_from_dir(&mut self, dir: &PathBuf) -> Result<usize, LoadError>

Example:

let result = instance.extend_from_dir("value")?;

Parameters:

Name	Type	Required	Description
`dir`	`&PathBuf`	Yes	The dir

Returns: usize

Errors: Returns Err(LoadError).

RehydrationMap

Token → original PII text.

Relationship

A relationship between two elements in the document.

During extraction, targets may be unresolved keys (RelationshipTarget::Key). The derivation step resolves these to indices using the element anchor index.

Field	Type	Default	Description
`source`	`u32`	—	Index of the source element in `InternalDocument::elements`.
`target`	`RelationshipTarget`	—	Target of the relationship (resolved index or unresolved key).
`kind`	`RelationshipKind`	—	Semantic kind of the relationship.

Renderer

Trait for document renderers that convert extraction results to output strings.

Renderers are typically stateless converters that transform extracted content into a specific output format (Markdown, HTML, Djot, plain text, etc.). They participate in the standard Plugin lifecycle so custom renderers can be registered from any supported binding language.

The format name is exposed via Plugin::name. For stateless renderers the Plugin lifecycle methods (version, initialize, shutdown) all take no-op defaults and need not be overridden.

Thread Safety

Renderers must be Send + Sync (inherited from Plugin).

Methods

render_result()

Binding-safe rendering entry point for foreign-language plugin bridges.

Accepts one public extraction result and returns the rendered output.

Signature:

pub fn render_result(&self, result: &ExtractedDocument) -> Result<String, XbergError>

Example:

let result = instance.render_result(&todo!())?;

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: String

Errors: Returns Err(XbergError).

RendererRegistry

Registry for document renderer plugins.

Manages renderers that convert internal pipeline documents to output format strings.

Thread Safety

The registry is thread-safe and can be accessed concurrently from multiple threads.

Methods

default()

Signature:

pub fn default() -> RendererRegistry

Example:

let result = RendererRegistry::default();

Returns: RendererRegistry

RerankedDocument

A single document returned by the reranker, with its position in the input and score.

index maps back to the caller’s original document list, so metadata arrays (e.g. IDs, paths) can be reordered without passing them through the reranker.

Field	Type	Default	Description
`index`	`usize`	—	Position of this document in the original input `documents` slice.
`score`	`f32`	—	Relevance score in `[0, 1]`. Higher means more relevant to the query.
`document`	`String`	—	The document text.

RerankerBackend

Trait for in-process reranker backend plugins.

Cross-encoders score (query, document) pairs jointly and return a raw logit per document. The crate-level rerank dispatcher applies sigmoid to convert logits to [0, 1] scores, sorts descending by score, and truncates to top_k.

Async to match the convention used by EmbeddingBackend and other plugin traits. Host-language bridges wrap their synchronous host callables in spawn_blocking or the equivalent.

Thread safety

Backends must be Send + Sync + 'static. They are stored in Arc<dyn RerankerBackend> and may be called concurrently from xberg’s dispatcher. If the backend’s underlying model is not thread-safe, the backend itself must serialize access internally (e.g. via Mutex<Inner>).

Contract

rerank(query, documents) MUST return exactly documents.len() scores. The dispatcher validates this before sorting and returning to callers; a non-conforming backend surfaces as a XbergError::Validation, not a panic.
Scores are raw logits in any range — callers must NOT assume [0, 1]. The dispatcher applies sigmoid before sorting.
rerank may be called from any thread. Its future must be Send (enforced by async_trait when #[async_trait] is used on non-WASM targets).
shutdown() (inherited from Plugin) may be invoked concurrently with an in-flight rerank() call. Implementations must tolerate this — letting in-flight calls finish via the Arc reference and only releasing shared state that isn’t needed by rerank.

Runtime

The synchronous rerank entry uses tokio::task::block_in_place to await the trait’s async rerank, which requires a multi-thread tokio runtime. Callers running inside a current_thread runtime must use rerank_async instead.

Methods

rerank()

Score a list of documents against a query.

Returns one raw logit per document in the same order as the input. The dispatcher applies sigmoid to convert to [0, 1] scores.

Errors:

Implementations should return Plugin for backend-specific failures. The dispatcher validates the returned length against documents.len() before sorting.

Signature:

pub async fn rerank(&self, query: String, documents: Vec<String>) -> Result<Vec<f32>, XbergError>

Example:

let result = instance.rerank("value", vec![]).await?;

Parameters:

Name	Type	Required	Description
`query`	`String`	Yes	The query
`documents`	`Vec<String>`	Yes	The documents

Returns: Vec<f32>

Errors: Returns Err(XbergError).

RerankerBackendRegistry

Registry for reranker backend plugins.

Unlike OcrBackendRegistry, no default backends are registered — reranker backends are always supplied by the host language at runtime.

Unlike EmbeddingBackendRegistry, there is no cached dimension — cross-encoders return a single scalar per (query, document) pair, not a vector of fixed length.

Methods

default()

Signature:

pub fn default() -> RerankerBackendRegistry

Example:

let result = RerankerBackendRegistry::default();

Returns: RerankerBackendRegistry

RerankerConfig

Configuration for the reranking pipeline.

Controls which model to use, how many results to return, and download/cache behavior for local ONNX models.

Field	Type	Default	Description
`model`	`RerankerModelType`	`RerankerModelType::Preset { name: "balanced" }`	The reranker model to use (defaults to “balanced” preset if not specified).
`top_k`	`Option<usize>`	`None`	Return at most this many documents. `None` returns all. Applied after sorting by score, so the highest-scoring documents are kept.
`batch_size`	`usize`	`32`	Batch size for local ONNX cross-encoder inference.
`show_download_progress`	`bool`	`false`	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at `info` level on the `xberg::model_download` target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by `RerankerModelType::Llm` and `RerankerModelType::Plugin`, which download no model.
`cache_dir`	`Option<PathBuf>`	`None`	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for the reranker ONNX model. Controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for local inference. Defaults to `None` (auto-select per platform).
`max_rerank_duration_secs`	`Option<u64>`	`60`	Maximum wall-clock duration (in seconds) for a single `rerank()` call when using `RerankerModelType::Plugin`. Applies only to the in-process plugin path — protects against hung host-language backends. On timeout, the dispatcher returns `Plugin` instead of blocking forever. `None` disables the timeout. The default (60 seconds) is conservative for common in-process inference; increase for large document sets on slow hardware.

Methods

default()

Signature:

pub fn default() -> RerankerConfig

Example:

let result = RerankerConfig::default();

Returns: RerankerConfig

RerankerEngine

Cross-encoder reranking model with thread-safe inference.

The rerank() method takes &self instead of &mut self, allowing it to be shared across threads via Arc<RerankerEngine> without mutex contention.

Supports two scoring heads, selected by head:

RerankerHead::CrossEncoder (default) — the original single-logit path.
RerankerHead::Qwen3Generative — Qwen3 generative-reranker path; requires true_token_id / false_token_id to have been resolved from the tokenizer.

ResolvedPreset

A preset merged with caller-supplied overrides (custom schema, prompt suffix, context map). Output is what the pipeline orchestrator consumes.

Field	Type	Default	Description
`id`	`String`	—	Source preset identifier.
`version`	`String`	—	Source preset version.
`fingerprint`	`String`	—	Fingerprint of the source preset file, used as a cache token.
`schema_name`	`String`	—	Schema name forwarded to the LLM.
`schema`	`serde_json::Value`	—	Effective JSON Schema (caller override or the preset’s own).
`system_prompt`	`String`	—	System prompt with rendered context appended.
`merge_mode`	`MergeMode`	—	Merge strategy for paginated outputs.
`preferred_call_mode`	`CallMode`	—	Preferred call mode.
`emit_citations`	`bool`	—	Whether the prompt asks for per-field citations.

ResolvedRecModel

Resolved recognition model with engine pool key for sharing.

Field	Type	Default	Description
`model_dir`	`PathBuf`	—	Exact path to the recognition ONNX model in the Hugging Face snapshot.
`dict_file`	`PathBuf`	—	Path to the character dictionary file.
`model_key`	`String`	—	Engine pool key for sharing engines across script families. Multiple families may share the same key (e.g. chinese and japanese both map to “v2:unified_server” when using server tier).

ResolvedStyle

Fully resolved (flattened) style after walking the inheritance chain.

Field	Type	Default	Description
`paragraph_properties`	`ParagraphProperties`	—	Fully-inherited paragraph formatting properties.
`run_properties`	`RunProperties`	—	Fully-inherited run formatting properties.

RevisionDelta

The content changes that make up a single revision.

For insertions and deletions the content field carries the added/removed lines as DiffLine::Added / DiffLine::Removed entries. For format changes, property_changes carries normalized before/after formatting values when the source document exposes them.

Field	Type	Default	Description
`content`	`Vec<DiffLine>`	`vec![]`	Line-level content changes for this revision.
`table_changes`	`Vec<CellChange>`	`vec![]`	Cell-level table changes for this revision.
`property_changes`	`Vec<PropertyChange>`	`vec![]`	Formatting or metadata property changes for this revision.

RowProperties

Row-level properties from <w:trPr>.

Field	Type	Default	Description
`height`	`Option<i32>`	`Default::default()`	Row height in twips.
`height_rule`	`Option<String>`	`Default::default()`	Height rule: `"auto"`, `"atLeast"`, or `"exact"`.
`is_header`	`bool`	—	Whether this row acts as a repeating table header.
`cant_split`	`bool`	—	Whether this row may be split across a page break.

RstExtractor

Native Rust reStructuredText extractor.

Parses RST documents using document tree parsing and extracts:

Metadata from field lists
Document structure (headings, sections)
Text content and inline formatting
Code blocks and directives
Tables and lists

Methods

default()

Signature:

pub fn default() -> RstExtractor

Example:

let result = RstExtractor::default();

Returns: RstExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

RtDetrModel

Docling RT-DETR v2 layout detection model.

This model is NMS-free (transformer-based end-to-end detection).

Input tensors:

images: f32 [batch, 3, 640, 640] (preprocessed pixel data)
orig_target_sizes: i64 [batch, 2] ([height, width] of source image)

Output tensors:

labels: i64 [batch, num_queries] (class IDs, 0-16)
boxes: f32 [batch, num_queries, 4] (bounding boxes in original image coordinates)
scores: f32 [batch, num_queries] (confidence scores)

Methods

detect()

Signature:

pub fn detect(&mut self, img: &RgbImage) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect(&todo!())?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_with_threshold()

Signature:

pub fn detect_with_threshold(&mut self, img: &RgbImage, threshold: f32) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect_with_threshold(&todo!(), 0.5)?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img
`threshold`	`f32`	Yes	The threshold

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_batch()

Signature:

pub fn detect_batch(&mut self, images: &Vec<RgbImage>, threshold: Option<f32>) -> Result<Vec<Vec<LayoutDetection>>, LayoutError>

Example:

let result = instance.detect_batch(&todo!(), Some(0.5))?;

Parameters:

Name	Type	Required	Description
`images`	`&Vec<RgbImage>`	Yes	The images
`threshold`	`Option<f32>`	No	The threshold

Returns: Vec<Vec<LayoutDetection>>

Errors: Returns Err(LayoutError).

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

RtfExtractor

Native Rust RTF extractor.

Extracts text content, metadata, and structure from RTF documents

Methods

default()

Signature:

pub fn default() -> RtfExtractor

Example:

let result = RtfExtractor::default();

Returns: RtfExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Run

A formatted text run within a DOCX paragraph.

Field	Type	Default	Description
`text`	`String`	—	Plain text content of this run.
`bold`	`bool`	—	Bold formatting flag.
`italic`	`bool`	—	Italic formatting flag.
`underline`	`bool`	—	Underline formatting flag.
`strikethrough`	`bool`	—	Strikethrough formatting flag.
`subscript`	`bool`	—	Subscript vertical alignment flag.
`superscript`	`bool`	—	Superscript vertical alignment flag.
`font_size`	`Option<u32>`	`Default::default()`	Font size in half-points (from `w:sz`).
`font_color`	`Option<String>`	`Default::default()`	Font color as “RRGGBB” hex (from `w:color`).
`highlight`	`Option<String>`	`Default::default()`	Highlight color name (from `w:highlight`).
`hyperlink_url`	`Option<String>`	`Default::default()`	Hyperlink URL, if this run is wrapped in a `<w:hyperlink>`.
`math_latex`	`Option<(String, bool)>`	`Default::default()`	LaTeX math content: (latex_source, is_display_math). When set, this run represents an equation and `text` is ignored.

RunProperties

Run-level formatting properties (bold, italic, font, size, color, etc.).

All fields are Option so that inheritance resolution can distinguish “not set” (None) from “explicitly set” (Some).

Field	Type	Default	Description
`bold`	`Option<bool>`	`Default::default()`	Bold formatting.
`italic`	`Option<bool>`	`Default::default()`	Italic formatting.
`underline`	`Option<bool>`	`Default::default()`	Underline formatting.
`strikethrough`	`Option<bool>`	`Default::default()`	Strikethrough formatting.
`color`	`Option<String>`	`Default::default()`	Hex RGB color, e.g. `"2F5496"`.
`font_size_half_points`	`Option<i32>`	`Default::default()`	Font size in half-points (`w:sz` val). Divide by 2 to get points.
`font_ascii`	`Option<String>`	`Default::default()`	ASCII font family (`w:rFonts w:ascii`).
`font_ascii_theme`	`Option<String>`	`Default::default()`	ASCII theme font (`w:rFonts w:asciiTheme`).
`vert_align`	`Option<String>`	`Default::default()`	Vertical alignment: “superscript”, “subscript”, or “baseline”.
`font_h_ansi`	`Option<String>`	`Default::default()`	High ANSI font family (w:rFonts w:hAnsi).
`font_cs`	`Option<String>`	`Default::default()`	Complex script font family (w:rFonts w:cs).
`font_east_asia`	`Option<String>`	`Default::default()`	East Asian font family (w:rFonts w:eastAsia).
`highlight`	`Option<String>`	`Default::default()`	Highlight color name (e.g., “yellow”, “green”, “cyan”).
`caps`	`Option<bool>`	`Default::default()`	All caps text transformation.
`small_caps`	`Option<bool>`	`Default::default()`	Small caps text transformation.
`shadow`	`Option<bool>`	`Default::default()`	Text shadow effect.
`outline`	`Option<bool>`	`Default::default()`	Text outline effect.
`emboss`	`Option<bool>`	`Default::default()`	Text emboss effect.
`imprint`	`Option<bool>`	`Default::default()`	Text imprint (engrave) effect.
`char_spacing`	`Option<i32>`	`Default::default()`	Character spacing in twips (from w:spacing w:val).
`position`	`Option<i32>`	`Default::default()`	Vertical position offset in half-points (from w:position w:val).
`kern`	`Option<i32>`	`Default::default()`	Kerning threshold in half-points (from w:kern w:val).
`theme_color`	`Option<String>`	`Default::default()`	Theme color reference (e.g., “accent1”, “dk1”).
`theme_tint`	`Option<String>`	`Default::default()`	Theme color tint modification (hex value).
`theme_shade`	`Option<String>`	`Default::default()`	Theme color shade modification (hex value).

SceptreOcrBackend

Sceptre-backed OCR implementation.

Reader construction resolves model files and creates inference sessions, so it is deferred until the first call for an effective configuration. The initialized reader is reused without holding the cache lock during inference; backend-wide admission prevents cached readers from oversubscribing the host.

Methods

default()

Signature:

pub fn default() -> SceptreOcrBackend

Example:

let result = SceptreOcrBackend::default();

Returns: SceptreOcrBackend

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_owned()

Signature:

pub async fn process_image_owned(&self, image_bytes: Vec<u8>, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_owned(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`Vec<u8>`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, language: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`language`	`&str`	Yes	The language

Returns: bool

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

confidence_semantics()

Sceptre’s page confidence is EasyOCR’s length-penalised custom_mean rescaled to 0-100. It is not comparable to Tesseract’s scale: on one document a real text page scored 39 while a pure line-art drawing scored 74, an inverted ordering relative to legibility. Never gate on this value.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

page_orientation_handling()

Measured on a /Rotate 270 scanned ordinance: sceptre produced character garbage on the sideways raster and only read correctly once the same page was rendered upright first. This is the same value as the trait default, made explicit here because it is a measurement, not an unverified inheritance.

Signature:

pub fn page_orientation_handling(&self) -> PageOrientationHandling

Example:

let result = instance.page_orientation_handling();

Returns: PageOrientationHandling

SceptreWasmEngine

Reusable, filesystem-free Sceptre reader for a single Gen2 recognizer group.

Construction verifies and compiles the supplied CRAFT and recognizer models. Browser hosts should create this object inside a Web Worker and reuse it.

SectionProperties

DOCX section properties parsed from w:sectPr element.

Field	Type	Default	Description
`page_width_twips`	`Option<i32>`	`Default::default()`	Page width in twips (from `w:pgSz w:w`).
`page_height_twips`	`Option<i32>`	`Default::default()`	Page height in twips (from `w:pgSz w:h`).
`orientation`	`Option<Orientation>`	`Default::default()`	Page orientation (from `w:pgSz w:orient`).
`margins`	`PageMargins`	—	Page margins (from `w:pgMar`).
`columns`	`ColumnLayout`	—	Column layout (from `w:cols`).
`doc_grid_line_pitch`	`Option<i32>`	`Default::default()`	Document grid line pitch in twips (from `w:docGrid w:linePitch`).

SecurityLimits

Configuration for security limits across extractors.

All limits are intentionally conservative to prevent DoS attacks while still supporting legitimate documents.

Field	Type	Default	Description
`max_archive_size`	`usize`	`524288000`	Maximum uncompressed size for archives (500 MB)
`max_compression_ratio`	`usize`	`100`	Maximum compression ratio before flagging as potential bomb (100:1)
`max_files_in_archive`	`usize`	`10000`	Maximum number of files in archive (10,000)
`max_nesting_depth`	`usize`	`1024`	Maximum nesting depth for structures (1024)
`max_entity_length`	`usize`	`1048576`	Maximum length of any single XML entity / attribute / token (1 MiB). This is a per-token cap, NOT a total cap — billion-laughs class attacks where a single entity expands to hundreds of MB are caught here, while normal long text content (a paragraph, a CDATA block) is caught by `max_content_size` instead.
`max_content_size`	`usize`	`104857600`	Maximum string growth and decoded image allocation per operation (100 MB). Per-page passes such as layout detection charge each batch against this limit, not the whole document; only `max_pages` bounds the rasters retained across a document (GH#1721).
`max_iterations`	`usize`	`10000000`	Maximum iterations per operation. For external redaction findings, this may lower but cannot raise the fixed 10,000-finding ceiling.
`max_xml_depth`	`usize`	`1024`	Maximum XML depth (1024 levels)
`max_table_cells`	`usize`	`100000`	Maximum aggregate table cells per document (100,000). Raise this for trusted large tabular inputs. Higher values permit proportionally more parsing work and output allocation.
`max_pages`	`Option<usize>`	`None`	Maximum number of pages (or slides, or frames) in a single document. `None` means unlimited. Checked once the count is known and before any per-page work (OCR, layout detection, rendering) starts. Byte-size limits do not bound page count: a scanned page can compress to a few kilobytes, so a document well under `max_content_size` or `max_archive_size` can still hold thousands of pages of per-page work. Defaults to `None` (unlimited) because a real ceiling here is workload-specific and a low default would silently reject legitimate large documents; callers that want a ceiling set this explicitly. Enforced for: PDF (`extractors::pdf`, page count via `xberg_native_pdf`/`lopdf`), PPTX (`extraction::pptx`, slide count from the archive’s slide parts), Keynote (`extractors::iwork::keynote`, slide count from `Index/Slide-*.iwa` entry names), ODP (`extractors::odp`, `draw:page` count in `content.xml`), and multi-frame TIFF images built with the `ocr` feature (`extractors::image`, frame count via the `tiff` crate). Not enforced for any other format, including DOCX, ODT, XLSX, legacy PPT/DOC, Pages/Numbers, and TIFF images when the `ocr` feature is disabled: those formats either have no fixed “page” the crate can count without doing the expensive work itself (DOCX/ODT page count is a layout outcome, not a stored value), or have no per-page pipeline to gate at all. Setting `max_pages` on a document of an unenforced format is silently a no-op, not a guarantee.

Methods

default()

Signature:

pub fn default() -> SecurityLimits

Example:

let result = SecurityLimits::default();

Returns: SecurityLimits

ServerConfig

API server configuration.

This struct holds all configuration options for the Xberg API server, including host/port settings, CORS configuration, and upload limits.

Defaults

host: “127.0.0.1” (localhost only)
port: 8000
cors_origins: empty vector (allows all origins)
max_request_body_bytes: 104_857_600 (100 MB)
max_multipart_field_bytes: 104_857_600 (100 MB)
job_timeout_secs: 600 (10 minutes)

Field	Type	Default	Description
`host`	`String`	`"127.0.0.1"`	Server host address (e.g., “127.0.0.1”, “0.0.0.0”)
`port`	`u16`	`8000`	Server port number
`cors_origins`	`Vec<String>`	`vec![]`	CORS allowed origins. Empty vector means allow all origins. If this is an empty vector, the server will accept requests from any origin. If populated with specific origins (e.g., `"<https://example.com>"`), only those origins will be allowed.
`max_request_body_bytes`	`usize`	`104857600`	Maximum size of request body in bytes (default: 100 MB)
`max_multipart_field_bytes`	`usize`	`104857600`	Maximum size of multipart fields in bytes (default: 100 MB)
`job_timeout_secs`	`u64`	`600`	Fallback timeout, in seconds, for `POST /extract-async` jobs whose request does not pin down `extraction_timeout_secs` (default: 600, 10 minutes). A per-request `extraction_timeout_secs: Some(n)` always overrides this value. An explicit `extraction_timeout_secs: null` deliberately does NOT mean “run unbounded” — it still falls back to this server-configured cap, because an unbounded job on a shared server is a denial-of-service risk.

Methods

default()

Signature:

pub fn default() -> ServerConfig

Example:

let result = ServerConfig::default();

Returns: ServerConfig

listen_addr()

Get the server listen address (host:port).

Signature:

pub fn listen_addr(&self) -> String

Example:

use xberg::core::ServerConfig;

let config = ServerConfig::default();
assert_eq!(config.listen_addr(), "127.0.0.1:8000");

Returns: String

cors_allows_all()

Check if CORS allows all origins.

Returns true if the cors_origins vector is empty, meaning all origins are allowed. Returns false if specific origins are configured.

Signature:

pub fn cors_allows_all(&self) -> bool

Example:

use xberg::core::ServerConfig;

let mut config = ServerConfig::default();
assert!(config.cors_allows_all());

config.cors_origins.push("<https://example.com>".to_string());
assert!(!config.cors_allows_all());

Returns: bool

is_origin_allowed()

Check if a given origin is allowed by CORS configuration.

Returns true if:

CORS allows all origins (empty origins list), or
The given origin is in the allowed origins list

Signature:

pub fn is_origin_allowed(&self, origin: &str) -> bool

Example:

use xberg::core::ServerConfig;

let mut config = ServerConfig::default();
assert!(config.is_origin_allowed("<https://example.com>"));

config.cors_origins.push("<https://allowed.com>".to_string());
assert!(config.is_origin_allowed("<https://allowed.com>"));
assert!(!config.is_origin_allowed("<https://denied.com>"));

Parameters:

Name	Type	Required	Description
`origin`	`&str`	Yes	The origin to check (e.g., “https://example.com”)

Returns: bool

max_request_body_mb()

Get maximum request body size in megabytes (rounded up).

Signature:

pub fn max_request_body_mb(&self) -> usize

Example:

use xberg::core::ServerConfig;

let mut config = ServerConfig::default();
assert_eq!(config.max_request_body_mb(), 100);

Returns: usize

max_multipart_field_mb()

Get maximum multipart field size in megabytes (rounded up).

Signature:

pub fn max_multipart_field_mb(&self) -> usize

Example:

use xberg::core::ServerConfig;

let mut config = ServerConfig::default();
assert_eq!(config.max_multipart_field_mb(), 100);

Returns: usize

SevenZExtractor

7z archive extractor.

Extracts file lists and text content from 7z archives.

Methods

default()

Signature:

pub fn default() -> SevenZExtractor

Example:

let result = SevenZExtractor::default();

Returns: SevenZExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

SharedModelPaths

Paths to shared models (detection + classification).

Field	Type	Default	Description
`det_model`	`PathBuf`	—	Exact path to the detection ONNX model in the Hugging Face snapshot.
`cls_model`	`PathBuf`	—	Exact path to the classification ONNX model in the Hugging Face snapshot.

SitemapUrl

A URL entry from a sitemap.

Field	Type	Default	Description
`url`	`String`	—	The URL.
`lastmod`	`Option<String>`	`Default::default()`	The last modification date, if present.
`changefreq`	`Option<String>`	`Default::default()`	The change frequency, if present.
`priority`	`Option<String>`	`Default::default()`	The priority, if present.

SlanetCell

A single cell detected by SLANeXT.

Field	Type	Default	Description
`polygon`	`Vec<f32>`	—	Bounding box polygon in image pixel coordinates. Format: [x1, y1, x2, y2, x3, y3, x4, y4] (4 corners, clockwise from top-left).
`bbox`	`Vec<f32>`	—	Axis-aligned bounding box derived from polygon: [left, top, right, bottom].
`row`	`usize`	—	Row index in the table (0-based).
`col`	`usize`	—	Column index within the row (0-based).

SlanetModel

SLANeXT table structure recognition model.

Wraps an ORT session for SLANeXT ONNX model and provides preprocessing, inference, and post-processing in a single recognize call.

SlanetResult

SLANeXT recognition result for a single table image.

Field	Type	Default	Description
`cells`	`Vec<SlanetCell>`	—	Detected cells with bounding boxes and grid positions.
`num_rows`	`usize`	—	Number of rows in the table.
`num_cols`	`usize`	—	Maximum number of columns across all rows.
`confidence`	`f32`	—	Average structure prediction confidence.
`structure_tokens`	`Vec<String>`	—	Raw HTML structure tokens (for debugging).

SparseEmbedding

A sparse learned embedding: vocabulary term indices and their weights.

indices are ascending vocabulary token ids; values[i] is the weight for indices[i]. The two arrays always have equal length. Only strictly-positive terms are retained, so the representation is genuinely sparse.

Field	Type	Default	Description
`indices`	`Vec<u32>`	—	Vocabulary token ids with non-zero weight, ascending.
`values`	`Vec<f32>`	—	Weights parallel to `SparseEmbedding::indices`.

SparseEmbeddingConfig

Configuration for the sparse-embedding pipeline.

Controls which model to use, batching, and download/cache behavior for the local ONNX SPLADE model.

Field	Type	Default	Description
`model`	`SparseEmbeddingModelType`	`SparseEmbeddingModelType::Preset`	The sparse-embedding model to use (defaults to the “opensearch-v3-distill” preset).
`batch_size`	`usize`	`16`	Batch size for local ONNX inference. SPLADE emits a `[seq, vocab]` logit tensor per document, so memory scales with batch size — keep this modest.
`max_length`	`usize`	`256`	Maximum token sequence length for the tokenizer.
`show_download_progress`	`bool`	`false`	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at `info` level on the `xberg::model_download` target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by `SparseEmbeddingModelType::Plugin`, which downloads no model.
`cache_dir`	`Option<PathBuf>`	`None`	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
`acceleration`	`Option<AccelerationConfig>`	`None`	Hardware acceleration for the sparse-embedding ONNX model.
`max_embed_duration_secs`	`Option<u64>`	`60`	Maximum wall-clock duration (in seconds) for a single embed call when using `SparseEmbeddingModelType::Plugin`. `None` disables the timeout.

Methods

default()

Signature:

pub fn default() -> SparseEmbeddingConfig

Example:

let result = SparseEmbeddingConfig::default();

Returns: SparseEmbeddingConfig

SparseEmbeddingEngine

SPLADE sparse-embedding model with thread-safe inference.

SparseEmbeddingPreset

Static metadata for a bundled SPLADE preset (WASM/Android-safe, no ORT).

Field	Type	Default	Description
`name`	`String`	—	Stable preset name referenced from config.
`model_repo`	`String`	—	HuggingFace repository hosting the ONNX model.
`model_file`	`String`	—	Path to the ONNX file within the repo.
`additional_files`	`Vec<String>`	—	Sibling files that must be downloaded alongside `model_file`.
`max_length`	`usize`	—	Maximum token sequence length.
`description`	`String`	—	Human-readable description.

SqliteExtractor

Extracts user tables from SQLite databases and GeoPackage containers.

Methods

default()

Signature:

pub fn default() -> SqliteExtractor

Example:

let result = SqliteExtractor::default();

Returns: SqliteExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

SsrfPolicy

SSRF policy configuration.

Field	Type	Default	Description
`deny_private`	`bool`	`true`	If true, reject URLs that resolve to private/metadata IP ranges.
`allowlist`	`Vec<HostMatcher>`	`vec![]`	Hostnames and IP ranges permitted regardless of `deny_private`, unless a matching address is in `denylist`. The allowlist is an override of `deny_private`, not an intersection with it. Precedence, in order: 1. A configured `denylist` CIDR always refuses a matching address. 2. `deny_private == false` permits addresses outside `denylist`; the allowlist is not consulted. 3. A hostname matching an `Exact` or `Suffix` entry is permitted before applying the built-in deny-list. When `denylist` is non-empty, it is still resolved so custom network denials can be enforced. 4. A literal or resolved IP inside a `Cidr` entry is permitted even though it is in the default deny-list. 5. Otherwise the default deny-list decides. An empty allowlist therefore denies nothing by itself — it simply leaves `deny_private` and the deny-list in sole control.
`denylist`	`Vec<HostMatcher>`	`vec![]`	IP ranges refused regardless of `deny_private` and `allowlist`. Only `HostMatcher::Cidr` entries are valid. Configured denials are checked before permissive settings, including against IPv4 addresses embedded in IPv6 and, on native targets, every address returned by DNS. They extend the built-in deny-list and cannot weaken it. A hostname is refused when an upstream proxy or remote browser performs the connection lookup, because that lookup cannot be bound to these checks.
`max_redirects`	`u8`	`5`	Maximum number of HTTP redirects to follow during validation.
`scheme_allowlist`	`Vec<String>`	`["http", "https"]`	Allowed URI schemes. Default: `["http", "https"]`. Only `http` and `https` are supported. An empty list denies every URL.

StaticEmbeddingEngine

A loaded static (model2vec) embedding model with thread-safe inference.

Rust-only: an opaque handle with no faithful binding representation (mirrors embeddings::engine::EmbeddingEngine / sparse_embeddings::engine::SparseEmbeddingEngine). Bindings drive inference through the module-level embed_texts dispatch, not this type directly.

StreamReader

Cursor-backed little-endian binary reader for HWP stream payloads.

StructuredData

Structured data (Schema.org, microdata, RDFa) block.

Field	Type	Default	Description
`data_type`	`StructuredDataType`	—	Type of structured data
`raw_json`	`String`	—	Raw JSON string representation
`schema_type`	`Option<String>`	`None`	Schema type if detectable (e.g., “Article”, “Event”, “Product”)

StructuredDataResult

Result of parsing a structured data file (JSON, JSONL, YAML, or TOML).

Field	Type	Default	Description
`content`	`String`	—	The extracted text content, formatted for readability.
`format`	`String`	—	The source format identifier (e.g. `"json"`, `"yaml"`, `"toml"`).
`metadata`	`HashMap<String, String>`	—	Key-value metadata extracted from recognized text fields.
`text_fields`	`Vec<String>`	—	JSON paths of fields that were classified as text-bearing.
`value`	`Option<serde_json::Value>`	`None`	The parsed document as a canonical `serde_json::Value` tree, when the source format could be represented as one. `None` only for TOML inputs whose `toml::Value` fails to round-trip through `serde_json::Value` (xberg-io/xberg#155): the extractor falls back to a raw code block in that case.
`flattened`	`Vec<String>`	—	Flattened `path: value` renderings for every leaf field, in traversal order. Previously computed and discarded (xberg-io/xberg#166); now surfaced so callers get a full-text view even when the structured renderer only emits headings/lists for a subset of fields.

StructuredExtractionConfig

Configuration for LLM-based structured data extraction.

Sends extracted document content to a VLM with a JSON schema, returning structured data that conforms to the schema.

Field	Type	Default	Description
`schema`	`serde_json::Value`	—	JSON Schema defining the desired output structure.
`schema_name`	`String`	`"extraction"`	Schema name passed to the LLM’s structured output mode.
`schema_description`	`Option<String>`	`/* serde(default) */`	Optional schema description for the LLM.
`strict`	`bool`	`/* serde(default) */`	Enable strict mode — output must exactly match the schema.
`prompt`	`Option<String>`	`/* serde(default) */`	Custom Jinja2 extraction prompt template. When `None`, a default template is used. Available template variables: - `{{ content }}` — The extracted document text. - `{{ schema }}` — The JSON schema as a formatted string. - `{{ schema_name }}` — The schema name. - `{{ schema_description }}` — The schema description (may be empty).
`llm`	`LlmConfig`	—	LLM configuration for the extraction.

Methods

default_schema_name()

Default Self::schema_name passed to the LLM’s structured-output mode.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_schema_name() -> String

Example:

let result = StructuredExtractionConfig::default_schema_name();

Returns: String

StructuredExtractor

Structured data extractor supporting JSON, JSONL/NDJSON, YAML, and TOML.

Methods

default()

Signature:

pub fn default() -> StructuredExtractor

Example:

let result = StructuredExtractor::default();

Returns: StructuredExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

StyleCatalog

Catalog of all styles parsed from word/styles.xml, plus document defaults.

Field	Type	Default	Description
`styles`	`HashMap<String, StyleDefinition>`	`HashMap::new()`	All named style definitions keyed by their style ID.
`default_paragraph_properties`	`ParagraphProperties`	—	Default paragraph formatting properties for the document.
`default_run_properties`	`RunProperties`	—	Default run formatting properties for the document.

StyleDefinition

A single style definition parsed from <w:style> in word/styles.xml.

Field	Type	Default	Description
`id`	`String`	—	The style ID (`w:styleId` attribute).
`name`	`Option<String>`	`None`	Human-readable name (`<w:name w:val="..."/>`).
`style_type`	`StyleType`	—	Style type: paragraph, character, table, or numbering.
`based_on`	`Option<String>`	`None`	ID of the parent style (`<w:basedOn w:val="..."/>`).
`next_style`	`Option<String>`	`None`	ID of the style to apply to the next paragraph (`<w:next w:val="..."/>`).
`is_default`	`bool`	—	Whether this is the default style for its type.
`paragraph_properties`	`ParagraphProperties`	—	Paragraph properties defined directly on this style.
`run_properties`	`RunProperties`	—	Run properties defined directly on this style.

StyledHtmlRenderer

Styled HTML renderer.

Implements the Renderer trait; registered as "html" when the html feature is active. Configuration is baked in at construction time — no per-render allocation for CSS resolution.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

render()

Signature:

pub fn render(&self, doc: &InternalDocument) -> Result<String, XbergError>

Example:

let result = instance.render(&todo!())?;

Parameters:

Name	Type	Required	Description
`doc`	`&InternalDocument`	Yes	The internal document

Returns: String

Errors: Returns Err(XbergError).

SubjectMatch

One vault match, in either direction of lookup.

Field	Type	Default	Description
`token`	`String`	—	Token
`original`	`String`	—	Original
`category`	`Option<String>`	`None`	Category parsed from the token’s bracket contents (e.g. `"EMAIL"` from `"[EMAIL_1]"`), or `None` if the token doesn’t follow the `"[CATEGORY_N]"` convention.

SummarizationConfig

Since: v1.0

Configuration for the summarisation post-processor.

Field	Type	Default	Description
`strategy`	`SummaryStrategy`	`SummaryStrategy::Extractive`	Summarisation strategy.
`max_tokens`	`Option<u32>`	`Default::default()`	Maximum summary length in tokens. `None` lets the backend pick a default.
`llm`	`Option<LlmConfig>`	`Default::default()`	LLM configuration for the abstractive backend. Ignored when `strategy = Extractive`. Required when `strategy = Abstractive`.

SummarizationProcessor

Post-processor that produces a DocumentSummary for the extraction result.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

estimated_duration_ms()

Signature:

pub fn estimated_duration_ms(&self, result: &ExtractedDocument) -> u64

Example:

let result = instance.estimated_duration_ms(&todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document

Returns: u64

SummaryInfo

Document metadata decoded from the OLE SummaryInformation-compatible property-set stream HWP names \x05HwpSummaryInformation (same binary layout as the \x05SummaryInformation stream used by Microsoft Office documents; see MS-OLEPS).

Field	Type	Default	Description
`title`	`Option<String>`	`Default::default()`	PIDSI_TITLE (property ID 2).
`subject`	`Option<String>`	`Default::default()`	PIDSI_SUBJECT (property ID 3).
`author`	`Option<String>`	`Default::default()`	PIDSI_AUTHOR (property ID 4).
`keywords`	`Option<String>`	`Default::default()`	PIDSI_KEYWORDS (property ID 5), stored verbatim (HWP does not split it into a list).
`comments`	`Option<String>`	`Default::default()`	PIDSI_COMMENTS (property ID 6).
`last_author`	`Option<String>`	`Default::default()`	PIDSI_LASTAUTHOR (property ID 8).
`created`	`Option<String>`	`Default::default()`	PIDSI_CREATE_DTM (property ID 12), rendered as an RFC 3339 UTC timestamp.
`modified`	`Option<String>`	`Default::default()`	PIDSI_LASTSAVE_DTM (property ID 13), rendered as an RFC 3339 UTC timestamp.

SupportedFormat

A supported document format entry.

Represents a file extension and its corresponding MIME type that Xberg can process.

Field	Type	Default	Description
`extension`	`String`	—	File extension (without leading dot), e.g., “pdf”, “docx”
`mime_type`	`String`	—	MIME type string, e.g., “application/pdf”

SvgOptions

SVG-specific configuration for the image-encode pipeline.

Applies when the source image is SVG or when the output format is set to ImageOutputFormat::Svg. Available when the svg feature is active.

Used via ImageExtractionConfig::svg.

Field	Type	Default	Description
`sanitize`	`bool`	`true`	Run SVG bytes through `usvg` sanitization (strips external `href` attributes, JavaScript event handlers, and `foreignObject` elements) even when the output format is `Native`. Defaults to `true`.
`render_dpi`	`f32`	`96`	Target DPI when rasterizing SVG to a pixel-based format (PNG, JPEG, WebP, HEIF). The tree’s viewBox is scaled by `render_dpi / 96.0` before the pixel buffer is allocated. Defaults to `96.0` (1× CSS pixel density).

Methods

default()

Signature:

pub fn default() -> SvgOptions

Example:

let result = SvgOptions::default();

Returns: SvgOptions

Table

Extracted table structure.

Represents a table detected and extracted from a document (PDF, image, etc.). Tables are converted to both structured cell data and Markdown format.

Field	Type	Default	Description
`cells`	`Vec<Vec<String>>`	`vec![]`	Table cells as a 2D vector (rows × columns)
`markdown`	`String`	—	Markdown representation of the table
`page_number`	`u32`	—	Page number where the table was found (1-indexed)
`bounding_box`	`Option<BoundingBox>`	`Default::default()`	Bounding box of the table’s position. Only populated when position data is available from the producing extractor. The coordinate space depends on how the table was produced, and callers must know which route produced a given `Table` before interpreting this field: - Tables extracted from a PDF’s native content, and tables recognized on a scanned PDF page that went through xberg’s OCR pipeline (`--force-ocr` / `--ocr-scanned-pages`), are in PDF points with a bottom-left origin (x0=left, y0=bottom, x1=right, y1=top; y increases upward). For the OCR case, the pipeline rescales the backend’s raw pixel output into this space before it reaches `Table::bounding_box` — see `rescale_ocr_bboxes_to_page_points` in `extractors::pdf::ocr`. - Tables detected by OCR on a standalone image with no backing PDF page (for example extracting a bare PNG/JPEG/TIFF) are in raster pixel coordinates with a top-left origin (x0=left, y0=top, x1=right, y1=bottom; y increases downward) — the same convention the OCR backend (Tesseract, PaddleOCR, candle-based backends) or the layout detector reported them in. There is no PDF page geometry to rescale into for this case, so the raw pixel box is passed through unchanged.
`table_id`	`Option<String>`	`Default::default()`	Stable identifier shared by every `tables[]` entry that represents a fragment of the same physical table. Assigned deterministically by the extraction pipeline (e.g. a sequential `"table-N"` in document order); never derived from randomness or wall-clock time, so the same input document always produces the same ids. Consumers can use it to reconcile the markdown blocks in `content` / `pages[].content` / `chunks[].content` with the structured entries in `tables[]`. `None` when the extractor did not assign one. Today, same-page fragments of one physical table are already merged into a single `tables[]` entry before ids are assigned (see PDF table stitching), so in practice `table_id` is unique per entry rather than shared across several. A table split across a page boundary is intentionally not linked — its per-page pieces get separate ids. Sharing one id across page-boundary fragments is a known possible future extension, not implemented yet.
`cell_styles`	`Vec<TableCellStyle>`	`vec![]`	Paragraph styles carried by individual cells, for the cells that have one. Sparse and flat on purpose. A DOCX banner row – row 0, one cell spanning the grid, styled `Heading1`..`Heading6` – is what Word’s navigation pane and a `TOC` field treat as the document outline, but as a table cell it reached consumers as anonymous text (GH#1587). `cells` keeps the bare text: prefixing it with `#` would put a markdown heading inside a table cell, which is invalid where it lands and would change text every existing consumer already reads. This list is the signal instead. Entries are only emitted for cells that actually carry a style, so an ordinary table serialises exactly as it did before. Indices are into `cells`.
`columns`	`Option<Vec<String>>`	`vec![]`	Header cells for this fragment, i.e. the first row of `cells`. Populated even when this fragment’s own header row was merged away or physically lives in a sibling fragment (see `table_id`), so a single fragment is interpretable in isolation. `None` when no header row could be determined.

Table2

A table reconstructed from OCR output (hOCR or TSV word bounding boxes).

Field	Type	Default	Description
`cells`	`Vec<Vec<String>>`	—	Cell text as a 2D grid (rows × columns).
`markdown`	`String`	—	Markdown-formatted table string.
`page_number`	`i32`	—	Zero-based page index this table was found on.

TableBorders

Borders for a table (6 borders: top, bottom, left, right, insideH, insideV).

Field	Type	Default	Description
`top`	`Option<BorderStyle>`	`Default::default()`	Top outer border.
`bottom`	`Option<BorderStyle>`	`Default::default()`	Bottom outer border.
`left`	`Option<BorderStyle>`	`Default::default()`	Left outer border.
`right`	`Option<BorderStyle>`	`Default::default()`	Right outer border.
`inside_h`	`Option<BorderStyle>`	`Default::default()`	Horizontal inner borders between rows.
`inside_v`	`Option<BorderStyle>`	`Default::default()`	Vertical inner borders between columns.

TableCell

Individual table cell with content and optional styling.

Future extension point for rich table support with cell-level metadata.

Field	Type	Default	Description
`content`	`String`	—	Cell content as text
`row_span`	`u32`	`1`	Row span (number of rows this cell spans)
`col_span`	`u32`	`1`	Column span (number of columns this cell spans)
`is_header`	`bool`	—	Whether this is a header cell

TableCellStyle

The paragraph style a single table cell’s text carries, located by grid position.

Flat rather than a nested Vec<Vec<Option<..>>>: the nested shape marshals badly across the FFI bindings, and the data is sparse anyway. See Table::cell_styles.

Field	Type	Default	Description
`row`	`u32`	—	Zero-indexed row of the cell this style belongs to.
`col`	`u32`	—	Zero-indexed column of the cell this style belongs to.
`heading_level`	`Option<u8>`	`Default::default()`	Outline level 1-6 when the style resolves to a heading, otherwise `None`.
`style_name`	`Option<String>`	`Default::default()`	Human-readable style name, e.g. `heading 2`.

TableClassifier

PP-LCNet table classifier model.

TableDiff

Cell-level changes for a pair of tables that share the same index.

Field	Type	Default	Description
`from_index`	`usize`	—	Zero-based index of the table in both `a.tables` and `b.tables`.
`to_index`	`usize`	—	Zero-based index in `b.tables` (equal to `from_index` for same-dimension tables).
`cell_changes`	`Vec<CellChange>`	—	Cell-level changes within the table.

TableGrid

Structured table grid with cell-level metadata.

Stores row/column dimensions and a flat list of cells with position info.

Field	Type	Default	Description
`rows`	`u32`	—	Number of rows in the table.
`cols`	`u32`	—	Number of columns in the table.
`cells`	`Vec<GridCell>`	`vec![]`	All cells in row-major order.

TableLook

Table look bitmask/flags controlling conditional formatting bands.

Field	Type	Default	Description
`first_row`	`bool`	—	Apply first-row conditional formatting.
`last_row`	`bool`	—	Apply last-row conditional formatting.
`first_column`	`bool`	—	Apply first-column conditional formatting.
`last_column`	`bool`	—	Apply last-column conditional formatting.
`no_h_band`	`bool`	—	Suppress horizontal banding.
`no_v_band`	`bool`	—	Suppress vertical banding.

TableProperties

Table-level properties from <w:tblPr>.

Field	Type	Default	Description
`style_id`	`Option<String>`	`Default::default()`	Style ID of the table style applied to this table.
`width`	`Option<TableWidth>`	`Default::default()`	Total table width specification.
`alignment`	`Option<String>`	`Default::default()`	Horizontal alignment: `"left"`, `"center"`, or `"right"`.
`layout`	`Option<String>`	`Default::default()`	Table layout algorithm: `"fixed"` or `"autofit"`.
`look`	`Option<TableLook>`	`Default::default()`	Conditional formatting flags for header/banded rows/columns.
`borders`	`Option<TableBorders>`	`Default::default()`	Table outer and inner border definitions.
`cell_margins`	`Option<CellMargins>`	`Default::default()`	Default cell margins applied to all cells in the table.
`indent`	`Option<TableWidth>`	`Default::default()`	Table indentation from the leading margin.
`caption`	`Option<String>`	`Default::default()`	Table caption text (from `<w:tblCaption>`).

TableTableGrid

Column widths from <w:tblGrid>.

Field	Type	Default	Description
`columns`	`Vec<i32>`	`vec![]`	Ordered list of column widths in twips.

TableWidth

Width specification used for tables and cells.

Field	Type	Default	Description
`value`	`i32`	—	Numeric width value; interpretation depends on `width_type`.
`width_type`	`String`	—	Width unit: `"dxa"` (twips), `"pct"` (50ths of a percent), `"auto"`, or `"nil"`.

TarExtractor

TAR archive extractor.

Extracts file lists and text content from TAR archives.

Methods

default()

Signature:

pub fn default() -> TarExtractor

Example:

let result = TarExtractor::default();

Returns: TarExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

TatrDetection

A single TATR detection result.

Field	Type	Default	Description
`bbox`	`Vec<f32>`	—	Bounding box in crop-pixel coordinates: `[x1, y1, x2, y2]`.
`confidence`	`f32`	—	Detection confidence score (0.0..1.0).
`class_name`	`TatrClass`	—	Detected class.

TatrModel

TATR (Table Transformer) table structure recognition model.

Wraps an ORT session for the TATR ONNX model and provides preprocessing, inference, and post-processing in a single recognize call.

TatrResult

Aggregated TATR recognition result with detections separated by class.

Field	Type	Default	Description
`rows`	`Vec<TatrDetection>`	—	Detected rows, sorted top-to-bottom by `y2`.
`columns`	`Vec<TatrDetection>`	—	Detected columns, sorted left-to-right by `x2`.
`headers`	`Vec<TatrDetection>`	—	Detected headers (ColumnHeader and ProjectedRowHeader).
`spanning`	`Vec<TatrDetection>`	—	Detected spanning cells.
`table_bbox`	`Option<Vec<f32>>`	`None`	The model’s own detected `Table` bbox (highest-confidence one), in crop-pixel coordinates. `None` if the model produced no `Table` detection above threshold (xberg-io/xberg#193).

TessdataManager

Manages tessdata file downloading, caching, and manifest generation.

TesseractBackend

Native Tesseract OCR backend.

This backend wraps the OcrProcessor and implements the OcrBackend trait, allowing it to be used through the plugin system.

Thread Safety

Uses Arc for shared ownership and is thread-safe (Send + Sync).

Lazy Initialization

The native Tesseract/Leptonica FFI handle is allocated on first use, not at backend construction. This allows the registry to be built without triggering expensive native initialization.

Methods

default()

Signature:

pub fn default() -> TesseractBackend

Example:

let result = TesseractBackend::default();

Returns: TesseractBackend

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_owned()

Signature:

pub async fn process_image_owned(&self, image_bytes: Vec<u8>, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_owned(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`Vec<u8>`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Probes for lang itself, not for English: with no config there is no other statement of intent, and a datapath holding only lang is exactly the layout GH#1891 is about. The answer is still read off the enumerated directory, so an absent language still reads false – probe_languages declines to resolve one that is nowhere on disk.

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

supports_language_for()

Probes with the config’s own effective languages, which is what a real job with this config resolves with (processor/execution.rs); a hardcoded eng made this deny a language the job loads without trouble whenever tessdata_path held only that language. GH#1891.

Signature:

pub fn supports_language_for(&self, config: &OcrConfig, language: &str) -> bool

Example:

let result = instance.supports_language_for(&todo!(), "value");

Parameters:

Name	Type	Required	Description
`config`	`&OcrConfig`	Yes	The configuration options
`language`	`&str`	Yes	The language

Returns: bool

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

supported_languages()

No config and no language to ask about, so this keeps probing the English datapath: there is nothing else to resolve with. supported_languages_for is the accurate answer for a caller that has a config.

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

supported_languages_for()

Signature:

pub fn supported_languages_for(&self, config: &OcrConfig) -> Vec<String>

Example:

let result = instance.supported_languages_for(&todo!());

Parameters:

Name	Type	Required	Description
`config`	`&OcrConfig`	Yes	The configuration options

Returns: Vec<String>

supports_table_detection()

Signature:

pub fn supports_table_detection(&self) -> bool

Example:

let result = instance.supports_table_detection();

Returns: bool

confidence_semantics()

Tesseract’s mean per-word classifier confidence, 0-100, is validated to track legibility: on a scanned ordinance, prose pages scored 89-95 and pure line-art drawings scored 36-62. It is safe to use as an absolute quality gate.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

page_orientation_handling()

Measured on a /Rotate 270 scanned ordinance: Tesseract reconstructs correct reading order on the sideways raster outright, with no upright-render step required.

Signature:

pub fn page_orientation_handling(&self) -> PageOrientationHandling

Example:

let result = instance.page_orientation_handling();

Returns: PageOrientationHandling

TesseractConfig

Tesseract OCR configuration.

Provides fine-grained control over Tesseract OCR engine parameters. Most users can use the defaults, but these settings allow optimization for specific document types (invoices, handwriting, etc.).

This is the public-facing counterpart of ocr::types::TesseractConfig (the internal, engine-facing representation with u8/String fields instead of i32/Vec<String>). They are two independent struct definitions bridged only by an explicit From<&TesseractConfig> for crate::ocr::types::TesseractConfig impl in ocr/types.rs — that conversion carries field values across, but each struct keeps its own Default impl, and the conversion does nothing to keep those two defaults in sync. Several production call sites (extractors::image::apply_default_tesseract_psm, configured_region_ocr, sparse_image_ocr_fallback_config) construct this struct’s default and convert it, bypassing the internal struct’s Default entirely — so if the two defaults disagree, standalone image OCR silently uses this struct’s value while PDF-embedded OCR (which can reach the internal Default directly when no tesseract_config is set) uses the other. When changing a default here, also update ocr::types::TesseractConfig::default, and vice versa.

Field	Type	Default	Description
`language`	`Vec<String>`	`["eng"]`	Language code(s) for OCR recognition. For Tesseract, languages are joined with “+”. A list is the canonical form and the only form accepted by the binding object APIs (Python, Node, PHP, WASM, etc.): `["eng", "deu"]`. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”).
`psm`	`Option<i32>`	`None`	Page Segmentation Mode (1-13). PSM 0 is rejected: Tesseract’s `PSM_OSD_ONLY` performs orientation and script detection with no character recognition, so it cannot satisfy a text-extraction request and previously yielded an empty document (GH#1586). `None` (the default) means the caller made no explicit choice: the extraction pipeline applies its own context-appropriate PSM (whole-image PSM 11, vertical- language PSM 5, layout-region PSM 6, or the sparse-text retry’s PSM 3) exactly as it would with no `TesseractConfig` at all — see issue #1573. Setting any other field on this struct no longer changes that behaviour. A rendered PDF page (`force_ocr` / `force_ocr_pages` / scanned-page OCR) that is one full-page scan gets the same whole-image PSM as standalone image OCR of that raster (11, or 5 for a vertical language), so the scan needs no explicit `psm` (GH#1786). A page that is not a scan, such as forced OCR of a vector page, keeps the engine’s automatic-layout PSM (3). Common explicit values: - 3: Fully automatic page segmentation (native engine default) - 6: Assume a single uniform block of text (WASM engine default — avoids layout-analysis hang) - 11: Sparse text with no particular order
`output_format`	`String`	`"markdown"`	Output format (“text” or “markdown”)
`oem`	`i32`	`3`	OCR Engine Mode (0-3). - 0: Legacy engine only - 1: Neural nets (LSTM) only (usually best) - 2: Legacy + LSTM - 3: Default (based on what’s available)
`min_confidence`	`f64`	`0`	Minimum confidence threshold (0.0-100.0). Words with confidence below this threshold may be rejected or flagged.
`preprocessing`	`Option<ImagePreprocessingConfig>`	`None`	Image preprocessing configuration. Controls how images are preprocessed before OCR. Can significantly improve quality for scanned documents or low-quality images.
`enable_table_detection`	`bool`	`true`	Enable automatic table detection and reconstruction
`table_min_confidence`	`f64`	`0`	Minimum confidence threshold for table detection (0.0-1.0)
`table_column_threshold`	`i32`	`50`	Column threshold for table detection (pixels)
`table_row_threshold_ratio`	`f64`	`0.5`	Row threshold ratio for table detection (0.0-1.0)
`use_cache`	`bool`	`true`	Enable OCR result caching
`classify_use_pre_adapted_templates`	`bool`	`true`	Use pre-adapted templates for character classification
`language_model_ngram_on`	`bool`	`true`	Enable N-gram language model. Kept on by default (see `Self::default` and `ocr::types::TesseractConfig::language_model_ngram_on` for the rationale); keep this field’s default in sync with the internal struct’s.
`tessedit_dont_blkrej_good_wds`	`bool`	`true`	Don’t reject good words during block-level processing
`tessedit_dont_rowrej_good_wds`	`bool`	`true`	Don’t reject good words during row-level processing
`tessedit_enable_dict_correction`	`bool`	`true`	Enable dictionary correction
`tessedit_char_whitelist`	`String`	`""`	Whitelist of allowed characters (empty = all allowed)
`tessedit_char_blacklist`	`String`	`""`	Blacklist of forbidden characters (empty = none forbidden)
`tessedit_use_primary_params_model`	`bool`	`true`	Use primary language params model
`textord_space_size_is_variable`	`bool`	`true`	Variable-width space detection
`thresholding_method`	`i32`	`0`	Tesseract image-binarization method (0-2): 0 = Otsu (default), 1 = LeptonicaOtsu, 2 = Sauvola. Sent to Tesseract’s integer `thresholding_method` engine variable. GH#1784: used to be a `bool` sent as `"true"`/`"false"`, silently ignored by Tesseract’s integer parser. A config for the old field still deserializes: `false` -> `0` (Otsu, the prior no-op), `true` -> `1` (LeptonicaOtsu, the old “adaptive” doc).

Methods

default()

Signature:

pub fn default() -> TesseractConfig

Example:

let result = TesseractConfig::default();

Returns: TesseractConfig

TesseractWasmBackend

WASM-compatible Tesseract OCR backend.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

confidence_semantics()

The WASM Tesseract backend reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

TextAnnotation

Inline text annotation — byte-range based formatting and links.

Annotations reference byte offsets into the node’s text content, enabling precise identification of formatted regions.

Field	Type	Default	Description
`start`	`u32`	—	Start byte offset in the node’s text content (inclusive).
`end`	`u32`	—	End byte offset in the node’s text content (exclusive).
`kind`	`AnnotationKind`	—	Annotation type.

TextExtractionResult

Plain text and Markdown extraction result.

Contains the extracted text along with statistics and, for Markdown files, structural elements like headers and links.

Field	Type	Default	Description
`content`	`String`	—	Extracted text content
`line_count`	`usize`	—	Number of lines
`word_count`	`usize`	—	Number of words
`character_count`	`usize`	—	Number of characters
`headers`	`Option<Vec<String>>`	`None`	Markdown headers (text only, Markdown files only)
`links`	`Option<Vec<MarkdownLink>>`	`None`	Markdown links (Markdown files only).
`code_blocks`	`Option<Vec<MarkdownCodeBlock>>`	`None`	Code blocks (Markdown files only).

TextMetadata

Text/Markdown metadata.

Extracted from plain text and Markdown files. Includes word counts and, for Markdown, structural elements like headers and links.

Field	Type	Default	Description
`line_count`	`u32`	—	Number of lines in the document
`word_count`	`u32`	—	Number of words
`character_count`	`u32`	—	Number of characters
`headers`	`Option<Vec<String>>`	`vec![]`	Markdown headers (headings text only, for Markdown files)
`links`	`Option<Vec<MarkdownLink>>`	`vec![]`	Markdown links (for Markdown files).
`code_blocks`	`Option<Vec<MarkdownCodeBlock>>`	`vec![]`	Code blocks (for Markdown files).

TextWindow

One inference window and where it starts in the source text.

Field	Type	Default	Description
`byte_offset`	`usize`	—	Byte offset of `text` within the source string.
`text`	`String`	—	The window’s own text.

Theme

Complete theme with color scheme and font scheme.

Field	Type	Default	Description
`name`	`String`	—	Theme name (e.g., “Office Theme”).
`color_scheme`	`Option<ColorScheme>`	`Default::default()`	Color scheme (12 standard colors).
`font_scheme`	`Option<FontScheme>`	`Default::default()`	Font scheme (major and minor fonts).

TokenCounter

Per-category running counter for RedactionStrategy::TokenReplace.

Methods

new()

Create a fresh counter with no previous state.

Signature:

pub fn new() -> TokenCounter

Example:

let result = TokenCounter::new();

Returns: TokenCounter

TokenReducer

Orchestrates the token-reduction pipeline: SIMD scanning, filter chain, and optional semantic clustering.

TokenReductionConfig

Configuration for the token-reduction pipeline.

Field	Type	Default	Description
`level`	`ReductionLevel`	`ReductionLevel::Moderate`	Reduction intensity level.
`language_hint`	`Option<String>`	`None`	ISO 639-1 language code hint for stopword selection (e.g. `"en"`, `"de"`).
`preserve_markdown`	`bool`	`false`	Preserve Markdown formatting tokens during reduction.
`preserve_code`	`bool`	`true`	Preserve code block contents unchanged.
`semantic_threshold`	`f32`	`0.3`	Cosine similarity threshold below which sentences are considered dissimilar.
`enable_parallel`	`bool`	`true`	Use Rayon parallel iterators for multi-core processing.
`use_simd`	`bool`	`true`	Use SIMD-optimized text scanning where available.
`custom_stopwords`	`Option<HashMap<String, Vec<String>>>`	`None`	Per-language custom stopword lists (`language_code → stopword_list`).
`preserve_patterns`	`Vec<String>`	`vec![]`	Regex patterns whose matched text is always preserved unchanged.
`target_reduction`	`Option<f32>`	`None`	Target fraction of text to retain (0.0–1.0); `None` = no fixed target.
`enable_semantic_clustering`	`bool`	`false`	Group semantically similar sentences and emit only one per cluster.
`preserve_important_words`	`bool`	`true`	Skip removal of words with “important” characteristics (all-caps acronyms, words containing digits, mixed-case identifiers, very long words) during the `Aggressive`/`Maximum` common-word removal pass. `true` (the default) protects those words even when they would otherwise be dropped as low-value filler; `false` lets the frequency/ length heuristics apply uniformly to every word, including ones that look like acronyms or technical terms (#269).

Methods

default()

Signature:

pub fn default() -> TokenReductionConfig

Example:

let result = TokenReductionConfig::default();

Returns: TokenReductionConfig

TokenReductionOptions

Token reduction configuration.

Field	Type	Default	Description
`mode`	`String`	`"off"`	Reduction mode: “off”, “light”, “moderate”, “aggressive”, “maximum”
`preserve_important_words`	`bool`	`true`	Preserve important words (capitalized, technical terms)

Methods

default()

Signature:

pub fn default() -> TokenReductionOptions

Example:

let result = TokenReductionOptions::default();

Returns: TokenReductionOptions

TokenizerBackend

Trait for in-process tokenizer backend plugins.

Unlike EmbeddingBackend, this trait is synchronous: the chunk splitter calls Self::count_tokens inside its boundary search, many times per chunk, so counting must be a direct call with no async dispatch. Host-language bridges (PyO3, napi-rs, etc.) invoke their host callable synchronously on the calling thread; implementations should keep count_tokens cheap — it dominates chunking time when the backend is slow.

Lifecycle

initialize() is called once during registration, before any count_tokens call; lazy-loading implementations should load their vocabulary there. After registration succeeds, count_tokens may be called from any thread, concurrently. shutdown() runs on unregistration and may overlap an in-flight count_tokens call from a chunking run that resolved the backend earlier — implementations must tolerate this, e.g. by keeping the resources count_tokens needs alive via Arc.

Thread safety

Backends must be Send + Sync + 'static (inherited from Plugin). They are stored in Arc<dyn TokenizerBackend> and called concurrently from xberg’s chunking pipeline. If the underlying tokenizer isn’t thread-safe, the backend must serialize access internally.

Contract

count_tokens must return a non-zero count for non-empty text. The registry probes this once at registration and rejects backends that report zero — a zero count would make every span appear to fit any budget. At runtime, a zero count for non-empty text is not trusted: the chunker substitutes the character count and logs the substitution. (An implementation may still return 0 for the empty string.)
count_tokens must not panic; return a best-effort count for text the tokenizer can’t fully process.
Counting should be deterministic for a given input — the splitter may evaluate overlapping spans of the same text repeatedly.

Methods

count_tokens()

Count the tokens in text according to this backend’s tokenizer.

Signature:

pub fn count_tokens(&self, text: &str) -> usize

Example:

let result = instance.count_tokens("value");

Parameters:

Name	Type	Required	Description
`text`	`&str`	Yes	The text

Returns: usize

TokenizerBackendRegistry

Registry for tokenizer backend plugins.

Like EmbeddingBackendRegistry, no default backends are registered — tokenizer backends are always supplied by the caller at runtime.

Unlike the embedding registry there is no cached shape metadata: a tokenizer backend is a pure &str -> usize counter, validated once at registration with a probe (see Self::register).

Methods

default()

Signature:

pub fn default() -> TokenizerBackendRegistry

Example:

let result = TokenizerBackendRegistry::default();

Returns: TokenizerBackendRegistry

TracingLayer

A tower::Layer that wraps each extraction in a semantic tracing span.

TranscriptionConfig

Configuration for audio/video transcription (speech-to-text).

When present and enabled, Xberg will route audio and video files (mp3, mp4, m4a, wav, webm, etc.) through the transcription pipeline.

The heavy dependencies (ORT, hf-hub, symphonia) are only pulled when the transcription feature is enabled. The config struct itself is available under transcription-types so that ExtractionConfig round-trips on all targets.

All fields have sensible defaults. The recommended starting point is:

[extraction.transcription]
enabled = true
model = "tiny"

Field	Type	Default	Description
`enabled`	`bool`	`true`	Master switch. When `false`, the transcription pipeline is not run. The extractor is registered for audio/video MIME types whenever the `transcription` feature is compiled in, independently of this flag, so an audio/video input with `enabled = false` fails with an `XbergError::Transcription` explaining how to turn transcription on — it does not fall through to another extractor.
`model`	`WhisperModel`	`WhisperModel::Tiny`	Whisper model size to use. Smaller = faster + lower memory. `tiny` is the pragmatic default for first-time users and CI.
`language`	`Option<String>`	`None`	Optional language hint (ISO-639-1 code, e.g. “en”, “de”). When `None` (default), the current engine falls back to English. For deterministic production output, always set this explicitly.
`timestamps`	`bool`	`false`	Whether to request segment-level timestamps. When `true`, the decoder prompt omits `<\|notimestamps\|>` so the model emits `<\|x.xx\|>` tokens, and each transcript segment becomes its own paragraph element carrying `start_ms` / `end_ms` attributes. When `false` (default), all segment text is joined into a single flat paragraph with no timing attributes.
`max_duration_ms`	`Option<u64>`	`1800000`	Hard safety limit on input duration (milliseconds). Files longer than this are rejected after decode, before model work. Default: 30 minutes. Set to `None` to disable (not recommended for untrusted input).
`max_bytes`	`Option<u64>`	`536870912`	Hard safety limit on input size (bytes). Default: 512 MiB. Protects against pathological or malicious uploads.
`timeout_ms`	`Option<u64>`	`600000`	Wall-clock timeout for the entire transcription operation (ms). Bounds audio decode, model resolution/download, and inference together. On expiry the extraction fails with an `XbergError::Transcription`. `None` disables the bound and lets the operation run unbounded (not recommended for untrusted input). Enforced on the async extraction path only; the size and duration caps (`max_bytes`, `max_duration_ms`) are checked on every path. Default: 10 minutes.
`model_cache_dir`	`Option<PathBuf>`	`None`	Optional alternate Hugging Face cache root for Whisper models. When unset, hf-hub follows `HF_HUB_CACHE`, `HUGGINGFACE_HUB_CACHE`, `HF_HOME`, XDG, and platform defaults. Files remain in the standard content-addressed snapshot layout and are not copied into an Xberg cache.
`allow_network`	`bool`	`true`	Allow network access to download models from Hugging Face Hub. When `false`, only previously cached models may be used. Useful for air-gapped or fully offline deployments.
`verify_hash`	`bool`	`false`	Request SHA256 verification of downloaded model files. Defaults to `false` because the resolver downloads from mutable Hugging Face refs unless callers pin and verify models out-of-band. Explicit `true` requests are rejected by the model resolver until pinned checksum metadata is available.

Methods

default()

Signature:

pub fn default() -> TranscriptionConfig

Example:

let result = TranscriptionConfig::default();

Returns: TranscriptionConfig

TranscriptionExtractor

The transcription extractor.

Priority is the normal default (50). If a user registers a custom higher-priority transcription backend via the plugin system, it will win.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Translation

Translation of the extracted content.

Holds the translated rendition of ExtractedDocument::content and (when preserve_markup was requested) the translated formatted_content. Chunks are translated in place inside ExtractedDocument::chunks[*].content rather than duplicated here.

Field	Type	Default	Description
`target_lang`	`String`	—	BCP-47 language tag the translation was produced into (e.g. `"de"`, `"fr-CA"`).
`source_lang`	`Option<String>`	`None`	BCP-47 source language. `None` when the translation backend was asked to detect.
`content`	`String`	—	Translated plain-text body. Matches the shape of `ExtractedDocument::content`.
`formatted_content`	`Option<String>`	`None`	Translated markup body (Markdown / HTML / etc.) when `preserve_markup` was enabled on the config. `None` otherwise.

TranslationConfig

Since: v1.0

Configuration for the translation post-processor.

Field	Type	Default	Description
`target_lang`	`String`	—	BCP-47 language tag for the target language (e.g. `"de"`, `"fr-CA"`).
`source_lang`	`Option<String>`	`None`	Optional explicit source language. `None` asks the backend to auto-detect.
`preserve_markup`	`bool`	`/* serde(default) */`	Translate the formatted (Markdown/HTML) rendition alongside plain text when `formatted_content` is present.
`llm`	`LlmConfig`	—	LLM configuration used for translation.

TranslationProcessor

Post-processor that translates content (and optionally formatted content and chunks) into the requested target language via the configured LLM.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process()

Signature:

pub async fn process(&self, result: &mut ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.process(&mut todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&mut ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: No return value.

Errors: Returns Err(XbergError).

processing_stage()

Signature:

pub fn processing_stage(&self) -> ProcessingStage

Example:

let result = instance.processing_stage();

Returns: ProcessingStage

should_process()

Signature:

pub fn should_process(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

let result = instance.should_process(&todo!(), &todo!());

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: bool

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

TreeSitterConfig

Configuration for tree-sitter language pack integration.

Controls grammar download behavior and code analysis options.

Example (TOML)

[tree_sitter]
languages = ["python", "rust"]
groups = ["web"]

[tree_sitter.process]
structure = true
comments = true
docstrings = true

Field	Type	Default	Description
`enabled`	`bool`	`true`	Enable code intelligence processing (default: true). When `false`, tree-sitter analysis is completely skipped even if the config section is present.
`cache_dir`	`Option<PathBuf>`	`None`	Custom cache directory for downloaded grammars. When `None`, uses the default: `~/.cache/tree-sitter-language-pack/v{version}/libs/`. Consumed both by the CLI (`tree-sitter download --from-config`, `cache warm`) and by `CodeExtractor` at extraction time, so that a configured cache directory is honoured wherever grammars are looked up or downloaded, not only during an explicit CLI download.
`languages`	`Option<Vec<String>>`	`None`	Languages to pre-download on init (e.g., `["python", "rust"]`). Consumed only by the CLI’s `tree-sitter download --from-config` and `cache warm` commands as a pre-download hint. Extraction itself does not read this field: a given source file always processes with a single, already auto-detected language, so there is nothing for a language allowlist to gate at extraction time.
`groups`	`Option<Vec<String>>`	`None`	Language groups to pre-download (e.g., `["web", "systems", "scripting"]`). Consumed only by the CLI’s `tree-sitter download --from-config` and `cache warm` commands, for the same reason as `languages` above.
`process`	`TreeSitterProcessConfig`	—	Processing options for code analysis.

Methods

default()

Signature:

pub fn default() -> TreeSitterConfig

Example:

let result = TreeSitterConfig::default();

Returns: TreeSitterConfig

TreeSitterProcessConfig

Processing options for tree-sitter code analysis.

Controls which analysis features are enabled when extracting code files.

Field	Type	Default	Description
`structure`	`bool`	`true`	Extract structural items (functions, classes, structs, etc.). Default: true.
`imports`	`bool`	`true`	Extract import statements. Default: true.
`exports`	`bool`	`true`	Extract export statements. Default: true.
`comments`	`bool`	`false`	Extract comments. Default: false.
`docstrings`	`bool`	`false`	Extract docstrings. Default: false.
`symbols`	`bool`	`false`	Extract symbol definitions. Default: false.
`diagnostics`	`bool`	`false`	Include parse diagnostics. Default: false.
`data_extraction`	`bool`	`false`	Extract a hierarchical key/value data tree from data-format files (JSON, YAML, TOML, XML, CSV, etc.). Default: false.
`chunk_max_size`	`Option<usize>`	`None`	Maximum chunk size in bytes. `None` disables chunking.
`content_mode`	`CodeContentMode`	`CodeContentMode::Chunks`	Content rendering mode for code extraction.

Methods

default()

Signature:

pub fn default() -> TreeSitterProcessConfig

Example:

let result = TreeSitterProcessConfig::default();

Returns: TreeSitterProcessConfig

TrocrBackend

TrOCR backend using candle transformers.

Recognizes text in images via Microsoft’s TrOCR model. Supports printed and handwritten text with four model variants (base/large × printed/handwritten).

Important: line-level only

TrOCR is trained to recognise a single line of text per image. When given a full-page document the model will typically decode only the most prominent region and produce nearly-empty output. Use TrOCR for cropped text regions — upstream stages must detect text regions (e.g. PaddleOCR text detector or a layout model) and hand each crop to this backend individually. For full-page VLM OCR use PaddleOcrVlBackend instead.

Configuration

TrOCR accepts backend options for runtime tuning:

{
  "variant": "base-printed",
  "device": "auto",
  "cache_dir": "/path/to/huggingface/cache",
  "hf_revision": "immutable-commit"
}

variant (string): "base-printed" (default), "large-printed", "base-handwritten", "large-handwritten"
device (string): "auto", "cpu", "cuda", "metal"
cache_dir (string, optional): explicit Hugging Face Hub cache root
hf_revision (string, optional): immutable model revision

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Recognize text in image_bytes via the configured TrOCR variant and device.

The variant is resolved by taking any explicit "variant" key from config.backend_options, falling back to the constructor-time variant stored in self.variant. Inference runs inside tokio::task::spawn_blocking so the async runtime is never blocked.

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

process_image_file()

Signature:

pub async fn process_image_file(&self, path: &PathBuf, config: &OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image_file("value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`config`	`&OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

supported_languages()

Signature:

pub fn supported_languages(&self) -> Vec<String>

Example:

let result = instance.supported_languages();

Returns: Vec<String>

backend_type()

Signature:

pub fn backend_type(&self) -> OcrBackendType

Example:

let result = instance.backend_type();

Returns: OcrBackendType

confidence_semantics()

TrOCR reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

TrocrBackendOptions

Runtime options accepted by the candle-trocr backend.

Field	Type	Default	Description
`variant`	`Option<CandleTrocrVariant>`	`Default::default()`	Optional model variant; the backend constructor’s variant is used when absent.
`device`	`Option<CandleDevicePreference>`	`Default::default()`	Optional per-call device override.
`cache_dir`	`Option<String>`	`Default::default()`	Optional Hugging Face cache root.
`hf_revision`	`Option<String>`	`Default::default()`	Optional immutable Hugging Face model revision.

TsvRow

Tesseract TSV row data for conversion.

This struct represents a single row from Tesseract’s TSV output format. TSV format includes hierarchical information (block, paragraph, line, word) along with bounding boxes and confidence scores.

Field	Type	Default	Description
`level`	`i32`	—	Hierarchical level (1=block, 2=para, 3=line, 4=word, 5=symbol)
`page_num`	`i32`	—	Page number (1-indexed)
`block_num`	`i32`	—	Block number within page
`par_num`	`i32`	—	Paragraph number within block
`line_num`	`i32`	—	Line number within paragraph
`word_num`	`i32`	—	Word number within line
`left`	`u32`	—	Left x-coordinate in pixels
`top`	`u32`	—	Top y-coordinate in pixels
`width`	`u32`	—	Width in pixels
`height`	`u32`	—	Height in pixels
`conf`	`f64`	—	Confidence score (0-100)
`text`	`String`	—	Recognized text

TypstExtractor

Typst document extractor

Methods

default()

Signature:

pub fn default() -> TypstExtractor

Example:

let result = TypstExtractor::default();

Returns: TypstExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_path()

Signature:

pub async fn extract_path(&self, path: &PathBuf, mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_path("value", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`path`	`&PathBuf`	Yes	Path to the file
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

UrlExtractionConfig

URL ingestion and crawl configuration.

Field	Type	Default	Description
`mode`	`UrlExtractionMode`	`UrlExtractionMode::Auto`	URL extraction mode.
`crawl`	`CrawlConfig`	`xberg::UrlExtractionConfig::default_xberg_crawl_config()`	Crawlberg crawl configuration used for HTTP(S) URL extraction.
`document_url_pattern`	`Option<String>`	`None`	Optional regex filter for document-discovered URLs.
`max_document_urls_per_result`	`Option<u32>`	`100`	Maximum URLs to follow per extraction result.
`max_total_urls`	`Option<u32>`	`1000`	Maximum URLs followed across the whole extraction call.
`allow_local_file_inputs`	`bool`	`true`	Allow bare local filesystem path inputs.
`allow_file_uris`	`bool`	`true`	Allow local `file://` URI inputs.

Methods

default()

Signature:

pub fn default() -> UrlExtractionConfig

Example:

let result = UrlExtractionConfig::default();

Returns: UrlExtractionConfig

default_xberg_crawl_config()

Default Self::crawl policy.

Public and on the type rather than a free private fn because generated bindings have to call it to reproduce the default, and a private one is out of their reach.

Signature:

pub fn default_xberg_crawl_config() -> CrawlConfig

Example:

let result = UrlExtractionConfig::default_xberg_crawl_config();

Returns: CrawlConfig

UserChunkConfig

User-provided chunk configuration.

Field	Type	Default	Description
`page_ranges`	`Option<Vec<PageRange>>`	`vec![]`	User-specified page ranges (overrides automatic chunking).
`pages_per_chunk`	`Option<u32>`	`Default::default()`	User-specified pages per chunk (overrides automatic calculation).
`force_chunking`	`bool`	—	Force chunking even for small documents.
`disable_chunking`	`bool`	—	Disable chunking even for large documents.

Validator

Trait for validator plugins.

Validators check extraction results for quality, completeness, or correctness. Unlike post-processors, validator errors fail fast - if a validator returns an error, the extraction fails immediately.

Use Cases

Quality Gates: Ensure extracted content meets minimum quality standards
Compliance: Verify content meets regulatory requirements
Content Filtering: Reject documents containing unwanted content
Format Validation: Verify extracted content structure
Security Checks: Scan for malicious content

Error Handling

Validator errors are fatal - they cause the extraction to fail and bubble up to the caller. Use validators for hard requirements that must be met.

For non-fatal checks, use post-processors instead.

Thread Safety

Validators must be thread-safe (Send + Sync).

Methods

validate()

Validate an extraction result.

Check the extraction result and return Ok(()) if valid, or an error if validation fails.

Returns:

Ok(()) if validation passes
Err(...) if validation fails (extraction will fail)

Errors:

XbergError::Validation - Validation failed
Any other error type appropriate for the failure

Example - Content Length Validation

Example - Quality Score Validation

Example - Security Validation

Signature:

pub async fn validate(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> Result<(), XbergError>

Example:

instance.validate(&todo!(), &todo!()).await?;

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extraction result to validate
`config`	`&ExtractionConfig`	Yes	Extraction configuration

Returns: No return value.

Errors: Returns Err(XbergError).

should_validate()

Optional: Check if this validator should run for a given result.

Allows conditional validation based on MIME type, metadata, or content. Defaults to true (always run).

Returns:

true if the validator should run, false to skip.

Signature:

pub fn should_validate(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool

Example:

/// Only validate PDF documents
fn should_validate(&self, result: &ExtractedDocument, config: &ExtractionConfig) -> bool {
    result.mime_type == "application/pdf"
}

Parameters:

Name	Type	Required	Description
`result`	`&ExtractedDocument`	Yes	The extracted document
`config`	`&ExtractionConfig`	Yes	The extraction config

Returns: bool

priority()

Optional: Get the validation priority.

Higher priority validators run first. Useful for ordering validation checks (e.g., run cheap validations before expensive ones).

Default priority is 50.

Returns:

Priority value (higher = runs earlier).

Signature:

pub fn priority(&self) -> i32

Example:

/// Run this validator first (it's fast)
fn priority(&self) -> i32 {
    100
}

Returns: i32

ValidatorRegistry

Registry for validator plugins.

Manages validators with priority-based execution order.

Methods

default()

Signature:

pub fn default() -> ValidatorRegistry

Example:

let result = ValidatorRegistry::default();

Returns: ValidatorRegistry

VersionResponse

Version response.

Field	Type	Default	Description
`version`	`String`	—	Xberg version string

VlmOcrBackend

VLM-based OCR backend using liter-llm vision models.

This backend sends images to a vision language model (e.g., GPT-4o, Claude) for text extraction, as an alternative to traditional OCR backends.

Methods

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

process_image()

Signature:

pub async fn process_image(&self, image_bytes: &[u8], config: &crate::OcrConfig) -> Result<ExtractedDocument, XbergError>

Example:

let result = instance.process_image(b"data", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`image_bytes`	`&\[u8\]`	Yes	The image bytes
`config`	`&crate::OcrConfig`	Yes	The configuration options

Returns: ExtractedDocument

Errors: Returns Err(XbergError).

supports_language()

Signature:

pub fn supports_language(&self, lang: &str) -> bool

Example:

let result = instance.supports_language("value");

Parameters:

Name	Type	Required	Description
`lang`	`&str`	Yes	The lang

Returns: bool

confidence_semantics()

VLM OCR reports no page-level confidence.

Signature:

pub fn confidence_semantics(&self) -> ConfidenceSemantics

Example:

let result = instance.confidence_semantics();

Returns: ConfidenceSemantics

WarmRequest

Cache warm request.

Field	Type	Default	Description
`all_embeddings`	`bool`	—	Download all embedding model presets
`embedding_model`	`Option<String>`	`Default::default()`	Specific embedding model preset to download
`ner`	`bool`	—	Download the default GLiNER NER model
`all_ner_models`	`bool`	—	Download every known GLiNER NER model
`ner_model`	`Option<String>`	`Default::default()`	Specific GLiNER NER model alias or catalog id to download

WarmResponse

Cache warm response.

Field	Type	Default	Description
`cache_dir`	`String`	—	Xberg-managed cache directory used. Hugging Face artifacts remain in the standard HF cache, and their resolved paths are included in `downloaded`.
`downloaded`	`Vec<String>`	—	Models that were downloaded
`already_cached`	`Vec<String>`	—	Models that were already cached

WebVttExtractor

WebVTT subtitle track extractor.

Emits one paragraph element per cue, carrying the cue’s timings and identifier as element attributes rather than as body text.

Methods

default()

Signature:

pub fn default() -> WebVttExtractor

Example:

let result = WebVttExtractor::default();

Returns: WebVttExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

WhisperEngine

Whisper ONNX inference engine.

Holds three sessions (encoder, decoder, decoder_with_past) and a tokenizer. Call WhisperEngine::transcribe to produce a transcript from PCM audio.

WhisperModelPaths

On-disk paths for all files needed to load a Whisper model in an ORT session.

Field	Type	Default	Description
`encoder`	`PathBuf`	—	Encoder ONNX model: `onnx/encoder_model.onnx`.
`decoder`	`PathBuf`	—	Decoder ONNX model (without KV-cache past): `onnx/decoder_model.onnx`. For sharded variants (Small+) this is the merged decoder.
`decoder_with_past`	`PathBuf`	—	Decoder ONNX model with KV-cache past: `onnx/decoder_with_past_model.onnx`. For sharded variants (Small+) this points to the same merged decoder as `decoder`.
`tokenizer`	`PathBuf`	—	`tokenizer.json` — vocabulary + BPE rules.
`config`	`PathBuf`	—	`config.json` — model hyper-parameters.
`n_mels`	`u32`	—	Number of mel filter banks expected by this model’s audio pre-processor. 80 for Tiny / Base / Small / Medium; 128 for LargeV3.

WordPerfectExtractor

Extractor for WordPerfect (.wpd) documents.

Supports WordPerfect 5.x/6.x (Windows) and Mac WordPerfect documents via the vendored libwpd/librevenge shim in xberg_libwpd.

Methods

default()

Signature:

pub fn default() -> WordPerfectExtractor

Example:

let result = WordPerfectExtractor::default();

Returns: WordPerfectExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

XlsxAppProperties

Application properties from docProps/app.xml for XLSX

Contains Excel-specific document metadata.

Field	Type	Default	Description
`application`	`Option<String>`	`Default::default()`	Application name (e.g., “Microsoft Excel”)
`app_version`	`Option<String>`	`Default::default()`	Application version
`doc_security`	`Option<i32>`	`Default::default()`	Document security level
`scale_crop`	`Option<bool>`	`Default::default()`	Scale crop flag
`links_up_to_date`	`Option<bool>`	`Default::default()`	Links up to date flag
`shared_doc`	`Option<bool>`	`Default::default()`	Shared document flag
`hyperlinks_changed`	`Option<bool>`	`Default::default()`	Hyperlinks changed flag
`company`	`Option<String>`	`Default::default()`	Company name
`worksheet_names`	`Vec<String>`	`vec![]`	Worksheet names

XmlExtractionResult

XML extraction result.

Contains extracted text content from XML files along with structural statistics about the XML document.

Field	Type	Default	Description
`content`	`String`	—	Extracted text content (XML structure filtered out)
`element_count`	`usize`	—	Total number of XML elements processed
`unique_elements`	`Vec<String>`	—	List of unique element names found (sorted)

XmlExtractor

XML extractor.

Extracts text content from XML files, preserving element structure information.

Methods

default()

Signature:

pub fn default() -> XmlExtractor

Example:

let result = XmlExtractor::default();

Returns: XmlExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_sync()

Signature:

pub fn extract_sync(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_sync(b"data", "value", &todo!())?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

XmlMetadata

XML metadata extracted during XML parsing.

Provides statistics about XML document structure.

Field	Type	Default	Description
`element_count`	`u32`	—	Total number of XML elements processed
`unique_elements`	`Vec<String>`	`vec![]`	List of unique element tag names (sorted)

YakeParams

YAKE-specific parameters.

Field	Type	Default	Description
`window_size`	`usize`	`2`	Window size for co-occurrence analysis (default: 2). Controls the context window for computing co-occurrence statistics.

Methods

default()

Signature:

pub fn default() -> YakeParams

Example:

let result = YakeParams::default();

Returns: YakeParams

YearRange

Year range for bibliographic metadata.

Field	Type	Default	Description
`min`	`Option<u32>`	`None`	Earliest (minimum) year in the range.
`max`	`Option<u32>`	`None`	Latest (maximum) year in the range.
`years`	`Vec<u32>`	`/* serde(default) */`	All individual years present in the collection.

YoloModel

YOLO-family layout detection model (YOLOv10, DocLayout-YOLO, YOLOX).

Methods

detect()

Signature:

pub fn detect(&mut self, img: &RgbImage) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect(&todo!())?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

detect_with_threshold()

Signature:

pub fn detect_with_threshold(&mut self, img: &RgbImage, threshold: f32) -> Result<Vec<LayoutDetection>, LayoutError>

Example:

let result = instance.detect_with_threshold(&todo!(), 0.5)?;

Parameters:

Name	Type	Required	Description
`img`	`&RgbImage`	Yes	The img
`threshold`	`f32`	Yes	The threshold

Returns: Vec<LayoutDetection>

Errors: Returns Err(LayoutError).

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

ZipBombValidator

Helper struct for validating ZIP archives for security issues.

ZipExtractor

ZIP archive extractor.

Extracts file lists and text content from ZIP archives.

Methods

default()

Signature:

pub fn default() -> ZipExtractor

Example:

let result = ZipExtractor::default();

Returns: ZipExtractor

name()

Signature:

pub fn name(&self) -> String

Example:

let result = instance.name();

Returns: String

version()

Signature:

pub fn version(&self) -> String

Example:

let result = instance.version();

Returns: String

initialize()

Signature:

pub fn initialize(&self) -> Result<(), XbergError>

Example:

instance.initialize()?;

Returns: No return value.

Errors: Returns Err(XbergError).

shutdown()

Signature:

pub fn shutdown(&self) -> Result<(), XbergError>

Example:

instance.shutdown()?;

Returns: No return value.

Errors: Returns Err(XbergError).

description()

Signature:

pub fn description(&self) -> String

Example:

let result = instance.description();

Returns: String

author()

Signature:

pub fn author(&self) -> String

Example:

let result = instance.author();

Returns: String

extract_content()

Signature:

pub async fn extract_content(&self, content: &[u8], mime_type: &str, config: &ExtractionConfig) -> Result<InternalDocument, XbergError>

Example:

let result = instance.extract_content(b"data", "value", &todo!()).await?;

Parameters:

Name	Type	Required	Description
`content`	`&\[u8\]`	Yes	The content to process
`mime_type`	`&str`	Yes	The mime type
`config`	`&ExtractionConfig`	Yes	The configuration options

Returns: InternalDocument

Errors: Returns Err(XbergError).

supported_mime_types()

Signature:

pub fn supported_mime_types(&self) -> Vec<String>

Example:

let result = instance.supported_mime_types();

Returns: Vec<String>

priority()

Signature:

pub fn priority(&self) -> i32

Example:

let result = instance.priority();

Returns: i32

Enums

ExecutionProviderType

ONNX Runtime execution provider type.

Determines which hardware backend is used for model inference. Auto (default) selects the best available provider per platform.

Value	Description
`Auto`	Auto-select: CoreML on macOS, CUDA on Linux, CPU elsewhere.
`Cpu`	CPU execution provider (always available).
`CoreMl`	Apple CoreML (macOS/iOS Neural Engine + GPU).
`Cuda`	NVIDIA CUDA GPU acceleration.
`TensorRt`	NVIDIA TensorRT (optimized CUDA inference).

CaptionAltTextMode

Since: v1.3

How a generated image caption interacts with existing alternate text.

Value	Description
`Preserve`	Keep existing alternate text, using the caption only when it is absent.
`Combine`	Join existing alternate text and the generated caption with `:` .
`Replace`	Use the generated caption even when alternate text is present.

MimeDetectionPolicy

Since: v1.1

Selects which evidence is authoritative when Xberg infers a MIME type.

Value	Description
`PreferContent`	Prefer a supported content signature, falling back to the filename extension.
`TrustExtension`	Trust a supported filename extension without reading content for detection. # Security Filenames are attacker-controlled input in uploads and downloads. Use this only for trusted sources; otherwise a misleading extension can route arbitrary content to the wrong extractor.
`ContentOnly`	Ignore the filename extension and require content-based detection.

ImageOutputFormat

Target format for re-encoding extracted images.

Controls whether and how extracted images are normalised to a uniform container format before being returned in ExtractedDocument.images. The default (Native) preserves the format produced by each extractor without any additional encode pass.

Callers that need uniform output — e.g. cloud pipelines that always store WebP thumbnails — set this once on ImageExtractionConfig.output_format rather than re-encoding downstream.

Serde shape

Uses a tagged enum: {"type": "native"}, {"type": "png"}, {"type": "jpeg", "quality": 90}, etc.

Value	Description
`Native`	Preserve whatever format the extractor produced (default). No re-encode pass is performed. `ExtractedImage.format` reflects the source format: JPEG for embedded PDF images, PNG for rasterised content, or the native container format from office documents.
`Png`	Re-encode all extracted images as PNG (lossless).
`Jpeg`	Re-encode all extracted images as JPEG at the given quality level. `quality` must be in `1..=100`. Values outside this range are clamped and a warning is emitted. Higher values produce larger files with less artefacting; 85 is a reasonable default. — Fields: `quality`: `u8`
`Webp`	Re-encode all extracted images as WebP at the given quality level. `quality` must be in `1..=100`. Values outside this range are clamped and a warning is emitted. 80 is a reasonable default. — Fields: `quality`: `u8`
`Heif`	Re-encode all extracted images as HEIF/HEIC at the given quality level. `quality` must be in `1..=100`. Values outside this range are clamped and a warning is emitted. 80 is a reasonable default. The encode path requires the `heic` feature; on builds without it, selecting this variant returns an `EncodeFailed` warning and leaves the image untouched. — Fields: `quality`: `u8`
`Svg`	Output pure-vector SVG. Lossless. Raster sources are not re-encoded (a warning is emitted and the image bytes are left untouched). When the source is already SVG, the bytes are passed through the `usvg` sanitizer (strips external hrefs, JS event handlers, and `foreignObject` elements) when `SvgOptions::sanitize` is `true`. Requires the `svg` feature.

ExtractInputKind

Source kind for ExtractInput.

Value	Description
`Bytes`	Raw in-memory bytes.
`Uri`	A filesystem path, `file://` URI, or HTTP(S) URL.

UrlExtractionMode

URL extraction mode.

Value	Description
`Auto`	Classify HTTP(S) resources after fetch.
`Document`	Treat the URI as a single remote document/page.
`Crawl`	Crawl from the seed URI and extract discovered pages/documents.

OutputFormat

Output format for extraction results.

Controls the format of the content field in ExtractedDocument. When set to Markdown, Djot, or Html, the output uses that format. Plain returns the raw extracted text.

Value	Description
`Plain`	Plain text content only (default)
`Markdown`	Markdown format
`Djot`	Djot markup format
`Html`	HTML format
`Json`	JSON tree format with heading-driven sections.
`DocTags`	Docling DocTags format (tables rendered as OTSL).
`Custom`	Custom renderer registered via the RendererRegistry. The string is the renderer name (e.g., “docx”, “latex”). — Fields: `0`: `String`

JupyterCellRendering

Controls how Jupyter notebook code cells are rendered during extraction.

A code cell carries both its source and any outputs that were saved in the notebook. Callers ingesting notebooks for AI agents want different slices of this depending on the task. Xberg never executes cells — Outputs and Both only surface outputs already stored in the .ipynb.

This toggle governs a code cell’s source body and its saved outputs. Markdown (prose) cells and structural markers (kernel language, cell id, tags, execution count) are unaffected — prose always renders and markers orient the reader regardless of mode.

Value	Description
`Source`	Render the code source as a fenced code block; omit saved outputs.
`Outputs`	Omit the code source; render only the saved cell outputs.
`Both`	Render both the code source and the saved outputs (default; preserves the historical behavior).

HtmlTheme

Built-in HTML theme selection.

Value	Description
`Default`	Sensible defaults: system font stack, neutral colours, readable line measure. CSS custom properties (`--kb-*`) are all defined so user CSS can override individual values.
`GitHub`	GitHub Markdown-inspired palette and spacing.
`Dark`	Dark background, light text.
`Light`	Minimal light theme with generous whitespace.
`Unstyled`	No built-in stylesheet emitted. CSS custom properties are still defined on `:root` so user stylesheets can reference `var(--kb-*)` tokens.

LateInteractionModelType

Late-interaction model types supported by Xberg.

Value	Description
`Preset`	Use a preset ColBERT model (recommended). — Fields: `name`: `String`
`Custom`	Use a custom ColBERT ONNX model from HuggingFace. — Fields: `model_id`: `String`, `model_file`: `String`, `additional_files`: `Vec<String>`, `max_length`: `i64`
`Plugin`	In-process late-interaction backend registered via the plugin system. — Fields: `name`: `String`

FormulaModel

Formula recognition model selection.

Value	Description
`LatexOcr`	RapidLaTeXOCR (MIT, pix2tex-derived): resizer + encoder + decoder ONNX, ~180 MB total, downloaded on demand.

TableModel

Which table structure recognition model to use.

Controls the model used for table cell detection within layout-detected table regions. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
`Tatr`	TATR (Table Transformer) – default, 30MB, DETR-based row/column detection.
`SlanetWired`	SLANeXT wired variant – 365MB, optimized for bordered tables.
`SlanetWireless`	SLANeXT wireless variant – 365MB, optimized for borderless tables.
`SlanetPlus`	SLANet-plus – 7.78MB, lightweight general-purpose.
`SlanetAuto`	Classifier-routed SLANeXT: auto-select wired/wireless per table. Uses PP-LCNet classifier (6.78MB) + both SLANeXT variants (730MB total).
`Disabled`	Disable table structure model inference entirely; use heuristic path only.

TableOverlapPreference

How to resolve overlapping native vs layout (TATR/SLANeXT) tables.

When both native detection and the layout table model produce a table for the same page region, one must be dropped. This controls which one wins. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
`Content`	Keep whichever table carries more content (cell count + markdown length). This is the historical default. TATR/SLANeXT tables usually recognize more cells and therefore win, which maximizes table-structure F1 but can lower text F1 when the recognized cell reflow diverges from the source reading order.
`Native`	Prefer the native table when it overlaps a layout table. Native tables preserve the source reading order, which scores higher on text F1 for documents where the layout model’s cell reflow diverges from the ground truth.
`Layout`	Prefer the layout (TATR/SLANeXT) table when it overlaps a native table.

LayoutStrategy

Which PDF pages the layout model runs on.

Layout detection renders each selected page to a raster and runs ONNX inference on it, which dominates extraction cost. This controls page selection; LayoutStrategy::Always preserves the historical behavior of running on every page. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value Description

Always Run layout detection unconditionally on every page.

Auto Pre-screen each page with cheap geometry signals and run the model only on pages likely to benefit (multi-column, table-bearing, figure-heavy, form-like, or rotated pages). Pages the pre-screen skips are processed exactly like pages where the model ran and found no regions. On the OCR path only inference is skipped; page rasters are still produced because OCR consumes them. For non-PDF inputs Auto behaves as LayoutStrategy::Always.

CredentialProviderConfig

Since: v1.1

Managed credential-provider configuration for OAuth2/STS-based authentication modes liter-llm cannot express via a static api_key. See LlmConfig::credential_provider.

Debug is implemented by hand: CredentialProviderConfig::AzureAd’s client_secret is a credential and must never be printed, matching LlmConfig’s own redaction policy. The other variants carry no secret material — CredentialProviderConfig::VertexOauth2 and CredentialProviderConfig::BedrockWebIdentity reference a file path, never the key or token itself.

Value	Description
`AzureAd`	Azure AD OAuth2 client-credentials flow (Azure OpenAI / Azure Cognitive Services). — Fields: `tenant_id`: `String`, `client_id`: `String`, `client_secret`: `String`, `scope`: `String`
`VertexOauth2`	Google Vertex AI OAuth2 via a service-account JSON key file on disk. Points at a file path rather than embedding the key inline: the key file contains an RSA private key — stronger secret material than an API key — and `LlmConfig` must never carry that directly, matching the credential-handling policy the rest of this module follows. — Fields: `service_account_key_file`: `String`, `scope`: `String`
`VertexAdc`	Google Vertex AI Application Default Credentials, resolved from the GCE/GKE/Cloud Run metadata server. Carries no secret material at all. — Fields: `scope`: `String`
`BedrockWebIdentity`	AWS STS `AssumeRoleWithWebIdentity` (EKS IRSA / OIDC federation) for Bedrock. — Fields: `role_arn`: `String`, `token_file`: `String`, `session_name`: `String`, `region`: `String`

CallMode

How a structured-extraction preset is dispatched to the model.

This is the preset-facing call mode (the preferred_call_mode field of a Preset). The structured pipeline has a richer runtime-only decision enum with skip and fallback states; this 3-variant type is the stable, serializable surface presets and bindings depend on.

Value	Description
`TextOnly`	Use the extracted text only.
`VisionOnly`	Use rasterized page images only.
`TextPlusVision`	Provide both extracted text and page images to the model.

MergeMode

How partial results from multiple model calls (e.g. per page batch) are combined.

Canonical home for the merge strategy referenced by presets and by the structured pipeline’s post-processing. There is intentionally only one merge type across the crate — do not introduce a second.

Value	Description
`ObjectMerge`	Deep-merge JSON objects field by field (later calls fill missing fields).
`ArrayConcat`	Concatenate top-level arrays across calls.
`ObjectFirst`	Keep the first non-empty result; ignore subsequent calls.

NerBackendKind

NER backend selector.

Value	Description
`Onnx`	`xberg-gliner` ONNX inference. Requires `ner-onnx` feature. Models download lazily from `xberg-io/gliner-models`.
`Llm`	liter-llm zero-shot NER via structured-output prompts. Requires `ner-llm` feature. Useful when domain-specific categories outstrip the ONNX taxonomy.

VlmFallbackPolicy

Policy controlling when VLM (Vision Language Model) OCR is used as a fallback.

This knob is syntactic sugar over the explicit OcrPipelineConfig stage ordering. When vlm_fallback is set and pipeline is None, an equivalent pipeline is synthesised at extraction time:

VlmFallbackPolicy::Disabled — no synthesis; single-backend mode (default).
VlmFallbackPolicy::OnLowQuality — tries the classical backend first; if the result scores below quality_threshold, tries VLM.
VlmFallbackPolicy::Always — skips the classical backend and sends every page to the VLM.

When OcrConfig::pipeline is explicitly set, vlm_fallback is ignored — the explicit pipeline takes precedence.

Errors:

Both OnLowQuality and Always require OcrConfig::vlm_config to be Some. Constructing an OcrConfig with one of these policies but no vlm_config is detected by OcrConfig::validate and will surface as a Validation error at extraction time, not a panic.

Value	Description
`Disabled`	No VLM fallback (default). Behaves identically to the pre-policy single-backend mode.
`OnLowQuality`	Try the classical OCR backend first. If the quality score is below `quality_threshold`, send the page to the VLM. `quality_threshold` is in the `[0.0, 1.0]` range, but it is not the same quantity reported on `score` (GH#1584). The accept decision blends text-shape quality with confidence, weighted 0.7/0.3 (`extractors::pdf::ocr::pipeline_stage_score`) – when the backend’s confidence is on a known scale it contributes only 30% of the compared score, so a page can clear this threshold on clean-looking text even while its own `PageOcrConfidence.score` reads below it. Do not calibrate this value by reading `PageOcrConfidence.score` off a sample page and expecting an equal `quality_threshold` to reproduce the same accept/reject outcome. A value of `0.5` is a reasonable starting point; calibrate with the Stage 0 benchmark harness. — Fields: `quality_threshold`: `f64`
`Always`	Skip the classical OCR backend entirely. Every page is sent to the VLM.

OcrStrategy

Which pages of a PDF get OCR’d when neither force_ocr nor force_ocr_pages applies.

Value Description

Auto OCR only when the native text layer fails a quality check (default). A scanner’s invisible OCR sidecar passes that check, so scanned pages carrying one are extracted natively. Use OcrStrategy::ScannedPages to OCR them instead.

ScannedPages Additionally OCR every page that looks like a scan. Pages are graded on raster coverage, whether the text layer is invisible or absent, the image codec, and the producer. Pages at or above min_confidence are OCR’d; the rest keep native text and still go through the Auto quality check. Detects that a text layer came from a scanner, not whether it is accurate, so a page carrying a good sidecar is OCR’d too. — Fields: min_confidence: f64

PdfBackend

PDF extraction backend selection.

Controls which engine parses and renders PDF documents. Wire format is snake_case in all serializers (JSON, TOML, YAML). Defaults to PdfBackend::Native – selecting anything else never changes behavior for a caller who does not opt in.

Value Description

Native xberg’s own pure-Rust PDF engine (default), crates/xberg-native-pdf.

Pdfium pdfium – Google’s PDFium engine, gated behind the pdf-pdfium Cargo feature (#700 added the selection level; #702 added the extraction engine, extractors::pdf::pdfium_engine, deliberately smaller in scope than Native – see that module’s doc comment for exactly what it extracts). A build without the pdf-pdfium feature rejects this at the CLI validation layer rather than silently falling back to Native. A build with the feature still enforces the selection at extraction time (extractors::pdf::PdfExtractor::extract_core), because that is the one dispatch point every caller – CLI, library use, API/MCP servers, language bindings – passes through; see that function’s doc comment for why the enforcement lives there and not only in CLI validation.

TableChunkingMode

Controls how markdown tables are handled when they exceed the chunk size limit.

Only applies when chunker_type is Markdown.

Variants

Split - Default behavior: tables are split at row boundaries like any other block element. Continuation chunks contain only data rows without the header, which can break downstream consumers that need column context.
RepeatHeader - Prepend the table header (header row + separator row) to every continuation chunk that contains data rows from the same table. Adds a small amount of duplicate text but ensures each chunk is self-contained for extraction, search, and LLM consumption.

Value	Description
`Split`	Split tables at row boundaries (default). Continuation chunks have no header.
`RepeatHeader`	Prepend the table header to every chunk that continues a split table.

ChunkerType

Type of text chunker to use.

Variants

Text - Generic text splitter, splits on whitespace and punctuation
Markdown - Markdown-aware splitter, preserves formatting and structure
Yaml - YAML-aware splitter, creates one chunk per top-level key
Semantic - Topic-aware chunker. With an EmbeddingConfig, splits at embedding-based topic shifts tuned by topic_threshold (default 0.75, lower = more splits). Without an embedding, falls back to a structural-boundary heuristic (ALL-CAPS headers, numbered sections, blank-line paragraphs) and merges groups into chunks capped at max_characters (default 1000). topic_threshold has no effect in the fallback path. For best results, pair with an embedding model.

Value	Description
`Text`	Generic whitespace- and punctuation-aware text splitter (default).
`Markdown`	Markdown-aware splitter that preserves heading and code-block boundaries.
`Yaml`	YAML-aware splitter that creates one chunk per top-level key.
`Semantic`	Topic-aware chunker that splits at embedding-based topic shifts.

ChunkSizing

How chunk size is measured.

Defaults to Characters (Unicode character count). When using token-based sizing, chunks are sized by token count according to the specified tokenizer.

Token-based sizing uses HuggingFace tokenizers loaded at runtime, or a tokenizer backend you register yourself. Any tokenizer available on HuggingFace Hub can be used, including OpenAI-compatible tokenizers (e.g., Xenova/gpt-4o, Xenova/cl100k_base). To size chunks with your own tokenizer instead (llama.cpp/GGUF vocabularies, SentencePiece models, custom vocabs), register a TokenizerBackend with register_tokenizer_backend and set model to the registered name.

Value	Description
`Characters`	Size measured in Unicode characters (default).
`Tokenizer`	Size measured in tokens from a HuggingFace tokenizer or a registered tokenizer backend. — Fields: `model`: `String`, `cache_dir`: `PathBuf`

EmbeddingModelType

Embedding model types supported by Xberg.

Value	Description
`Preset`	Use a preset model configuration (recommended) — Fields: `name`: `String`
`Custom`	Use a custom ONNX model from HuggingFace — Fields: `model_id`: `String`, `dimensions`: `usize`
`Llm`	Provider-hosted embedding model via liter-llm. Uses the model specified in the nested `LlmConfig` (e.g., `"openai/text-embedding-3-small"`). — Fields: `llm`: `LlmConfig`
`Plugin`	In-process embedding backend registered via the plugin system. The caller registers an `EmbeddingBackend` once (e.g. a wrapper around an already-loaded `llama-cpp-python`, `sentence-transformers`, or tuned ONNX model), then references it by name in config. Xberg calls back into the registered backend during chunking and standalone embed requests — no HuggingFace download, no ONNX Runtime requirement, no HTTP sidecar. When this variant is selected, only the following `EmbeddingConfig` fields apply: `normalize` (post-call L2 normalization) and `max_embed_duration_secs` (dispatcher timeout). Model-loading fields (`batch_size`, `cache_dir`, `show_download_progress`, `acceleration`) are ignored — the host owns the model lifecycle, so there is no download to report progress for. Semantic chunking falls back to `ChunkingConfig::max_characters` when this variant is used, since there is no preset to look a chunk-size ceiling up against — size your context window via `max_characters` directly. See `register_embedding_backend`. — Fields: `name`: `String`

RedactionOffsetEncoding

Since: v1.3

Unit that an external finding’s start / end offsets count in.

Value	Description
`Utf8Bytes`	UTF-8 byte offsets.
`UnicodeCodePoints`	Unicode scalar value (code point) offsets, as Presidio reports them.
`Utf16CodeUnits`	UTF-16 code unit offsets.

RerankerHead

Selects how a local ONNX reranker’s raw output tensor is turned into a score.

RerankerHead::CrossEncoder — classic single-logit cross-encoder head: the model emits [batch, 1] (or [batch]) logits; the caller applies sigmoid to get a [0, 1] score. This is the original, unchanged path.
RerankerHead::Qwen3Generative — Qwen3 generative-reranker head: the model emits [batch, seq, vocab] logits; the score is P("yes") read from the last token’s logits over the “yes”/“no” vocabulary entries, via a softmax over those two logits. Already a [0, 1] probability — no sigmoid is applied.

Value	Description
`CrossEncoder`	Single-logit cross-encoder head (sigmoid applied by the caller).
`Qwen3Generative`	Qwen3 generative-reranker head (softmax over yes/no token logits).

RerankerModelType

Reranker model types supported by Xberg.

Value	Description
`Preset`	Use a preset cross-encoder model (recommended). — Fields: `name`: `String`
`Custom`	Use a custom ONNX cross-encoder from HuggingFace. — Fields: `model_id`: `String`, `model_file`: `String`, `additional_files`: `Vec<String>`, `max_length`: `i64`, `head`: `RerankerHead`
`Llm`	Provider-hosted reranker via liter-llm (e.g. Cohere, Jina, Voyage). The model in the nested `LlmConfig` must be a rerank-capable model ID (e.g. `"cohere/rerank-english-v3.0"`). — Fields: `llm`: `LlmConfig`
`Plugin`	In-process reranker registered via the plugin system. The caller registers a `RerankerBackend` once (e.g. a wrapper around a `sentence-transformers` cross-encoder or a provider client), then references it by name in config. Xberg calls back into the registered backend — no HuggingFace download, no ONNX Runtime requirement. When this variant is selected, only `max_rerank_duration_secs` applies. Model-loading fields (`batch_size`, `cache_dir`, `show_download_progress`, `acceleration`) are ignored — the host owns the model lifecycle, so there is no download to report progress for. See `register_reranker_backend`. — Fields: `name`: `String`

SparseEmbeddingModelType

Sparse-embedding model types supported by Xberg.

Value	Description
`Preset`	Use a preset SPLADE model (recommended). — Fields: `name`: `String`
`Custom`	Use a custom SPLADE (`BertForMaskedLM`) ONNX model from HuggingFace. — Fields: `model_id`: `String`, `model_file`: `String`, `additional_files`: `Vec<String>`, `max_length`: `i64`
`Plugin`	In-process sparse-embedding backend registered via the plugin system. — Fields: `name`: `String`

WhisperModel

Supported Whisper model sizes.

These map to published ONNX exports on Hugging Face (onnx-community or similar orgs). The actual filenames and repos are resolved inside the transcription engine.

Value	Description
`Tiny`	Smallest, fastest, lowest quality. Good default for development and CI.
`Base`	Reasonable quality/speed tradeoff.
`Small`	Better accuracy with higher memory and cache use.
`Medium`	High quality; slower and more memory-intensive.
`LargeV3`	Best quality (large-v3). Use only when latency and memory use are acceptable.

CodeContentMode

Content rendering mode for code extraction.

Controls how extracted code content is represented in the content field of ExtractedDocument.

Value	Description
`Chunks`	Use TSLP semantic chunks as content (default).
`Raw`	Use raw source code as content.
`Structure`	Emit function/class headings + docstrings (no code bodies).

ListType

Type of list detection.

Value	Description
`Bullet`	Bullet points (-, *, •, etc.)
`Numbered`	Numbered lists (1., 2., etc.)
`Lettered`	Lettered lists (a., b., A., B., etc.)
`Indented`	Indented items

HwpError

Error type for HWP parsing.

Value	Description
`InvalidFormat`	The file does not match the HWP 5.0 format. — Fields: `0`: `String`
`UnsupportedVersion`	The HWP version or a feature is not supported (e.g. password-encrypted docs). — Fields: `0`: `String`
`Io`	An underlying I/O error occurred. — Fields: `0`: `String`
`Cfb`	A CFB compound-file error (stream not found, corrupt container, etc.). — Fields: `0`: `String`
`CompressionError`	Decompression of a zlib/deflate stream failed. — Fields: `0`: `String`
`ParseError`	The binary record stream could not be parsed. — Fields: `0`: `String`
`EncodingError`	A UTF-16LE string contained invalid data. — Fields: `0`: `String`
`NotFound`	A requested stream was not present in the compound file. — Fields: `0`: `String`

DrawingType

Whether the drawing is inline or anchored.

Value	Description
`Inline`	Drawing is inline: placed within the text flow at its insertion point.
`Anchored`	Drawing is anchored: floats at a fixed position relative to the page or paragraph. — Fields: `0`: `AnchorProperties`

WrapType

Text wrapping type around an anchored drawing.

Value	Description
`None`	No text wrapping; drawing floats above or below text.
`Square`	Text wraps in a square around the drawing bounding box.
`Tight`	Text wraps tightly to the drawing outline.
`TopAndBottom`	Text appears above and below but not to the sides.
`Through`	Text flows through the drawing’s transparent areas.

HeaderFooterType

Specifies which pages a header or footer applies to.

Value	Description
`Default`	Default header/footer (applies to all pages not covered by First or Even).
`First`	First-page header/footer.
`Even`	Even-page header/footer.
`Odd`	Odd-page header/footer.

NoteType

Distinguishes footnotes from endnotes in DOCX documents.

Value	Description
`Footnote`	Note appears at the bottom of the page.
`Endnote`	Note appears at the end of the document.

Orientation

Page orientation.

Value	Description
`Portrait`	Page height is greater than page width.
`Landscape`	Page width is greater than page height.

StyleType

The type of a style definition in DOCX.

Value	Description
`Paragraph`	Style applied to an entire paragraph (`<w:style w:type="paragraph">`).
`Character`	Style applied to a character run (`<w:style w:type="character">`).
`Table`	Style applied to a table (`<w:style w:type="table">`).
`Numbering`	Style applied to a list numbering definition (`<w:style w:type="numbering">`).

VerticalMerge

Vertical merge state for a table cell.

Value	Description
`Restart`	This cell starts a new vertically-merged group.
`Continue`	This cell continues a vertically-merged group started above.

ThemeColor

A theme color definition, either direct RGB or a system color with fallback.

Value	Description
`Rgb`	Direct hex RGB color (e.g., “156082”). — Fields: `0`: `String`
`System`	System color with fallback RGB (e.g., “windowText” with lastClr “000000”). — Fields: `name`: `String`, `last_color`: `String`

SecurityError

Security validation errors.

Value	Description
`ZipBombDetected`	Potential ZIP bomb detected — Fields: `compressed_size`: `u64`, `uncompressed_size`: `u64`, `ratio`: `f64`
`ArchiveTooLarge`	Archive exceeds maximum size — Fields: `size`: `u64`, `max`: `usize`
`TooManyFiles`	Archive contains too many files — Fields: `count`: `usize`, `max`: `usize`
`NestingTooDeep`	Nesting too deep — Fields: `depth`: `usize`, `max`: `usize`
`ContentTooLarge`	Content exceeds maximum size — Fields: `size`: `usize`, `max`: `usize`
`EntityTooLong`	Entity/string too long — Fields: `length`: `usize`, `max`: `usize`
`TooManyIterations`	Too many iterations — Fields: `count`: `usize`, `max`: `usize`
`XmlDepthExceeded`	XML depth exceeded — Fields: `depth`: `usize`, `max`: `usize`
`TooManyCells`	Aggregate table-cell limit exceeded. — Fields: `cells`: `usize`, `max`: `usize`
`TooManyPages`	Document has too many pages — Fields: `count`: `usize`, `max`: `usize`
`UnreadableEntry`	An archive entry could not be read, so its declared sizes could not be counted towards the archive limits. Reported rather than skipped: an unaccounted entry makes every aggregate total untrustworthy. — Fields: `index`: `usize`, `reason`: `String`

OcrBackendType

OCR backend types.

Value	Description
`Tesseract`	Tesseract OCR (native Rust binding)
`PaddleOcr`	PaddleOCR (Python-based, via FFI)
`Candle`	Candle-based VLM OCR (TrOCR, PaddleOCR-VL).
`Custom`	Name-selected built-in or third-party OCR backend.

ConfidenceSemantics

How a backend’s reported page-level confidence must be interpreted.

Backend confidence scores are not interchangeable. Tesseract’s mean word confidence is a classifier score validated to track legibility on a 0-100 scale. Sceptre (EasyOCR-based) reports a length-penalised custom_mean that is rescaled into the same 0-100 range but is not comparable — its ordering can be inverted relative to legibility (a dense prose page can score lower than a nearly-blank one). A page-rejection gate calibrated on Tesseract’s scale was once applied unconditionally to sceptre’s output and rejected every page of a 16-page document, emptying it. This descriptor exists so gating code can ask a backend what its number means instead of assuming.

Value	Description
`Legibility`	Validated to track legibility on a known scale — usable as an absolute quality gate. — Fields: `scale_max`: `f64`
`Uncalibrated`	A number is reported, but it is not validated to correlate with legibility. Never gate on it.
`None`	No page-level confidence is reported at all.

PageOrientationHandling

How a backend copes with a page raster whose text is not upright.

Rotated-page handling is a backend capability, not a universal guarantee. An A/B run this session against /Rotate 270 scanned pages showed the three handled cases genuinely differ: Tesseract reconstructs correct reading order on a sideways raster outright; PaddleOCR recognises the rotated text correctly (it warps each detected quad upright before running recognition) but leaves its block list in raw raster (y, x) order, so the caller must reorder; sceptre produces character garbage on the same sideways raster and only reads correctly once the page is rendered upright first. A caller that skips an upright-render step for a backend that actually needs one gets silent garbage, not an error.

Only one variant is discriminated (#657)

Read this before “simplifying” the type. There is exactly one decision point in the codebase that inspects this value: upright_raster_for_backend (crate::extractors::pdf::ocr), which tests orientation_handling != RequiresUpright and otherwise does nothing. Every other mention forwards the value to that test. So, to that codepath, SelfCorrecting and RecognisesRotatedText are behaviourally identical — the enum is a boolean at the point of use, and the three variants describe measured backend behaviour rather than three dispatch paths.

RecognisesRotatedText’s actual remedy is not this enum. The block-order fix is the backend_options["page_rotation_degrees"] hint injected by ocr_config_with_page_rotation_hint (crate::extractors::pdf::ocr) unconditionally, for every backend, which PaddleOcrBackend::process_image reads back (page_rotation_degrees_from_backend_options -> residual_rotation_for_reorder -> reorder_blocks_for_page_rotation, crate::paddle_ocr::backend) and applies internally. Declaring RecognisesRotatedText therefore changes nothing on its own; a backend in that class must also read the hint. Conversely, gating that hint on this enum would remove a field from Tesseract’s OcrConfig and hence from the OCR cache key (hash(image + language + config)), invalidating every cached page — do not do it without its own A/B.

PDF-route-only

Only the PDF OCR routes call OcrBackend::page_orientation_handling (the --force-ocr route via extract_with_ocr and the scanned-pages route via extract_mixed_ocr_native). The raw-image route (crate::extractors::image) never calls it: there is no /Rotate to consult, and orientation there is handled by the PP-LCNet document-orientation classifier (crate::doc_orientation) gated on OcrConfig::auto_rotate.

Cost of the default

The trait default is RequiresUpright (deliberately the least capable option, see OcrBackend::page_orientation_handling). A backend that does not declare therefore pays, on every page with /Rotate != 0, a re-encode plus rotation of the raster in upright_raster_for_backend and a bounding-box round-trip back through undo_upright_raster_correction. That is the safe direction to be wrong in, but it is not free, and for a backend that never got measured it is not known to be necessary either.

Value	Description
`SelfCorrecting`	Reconstructs reading order regardless of page rotation — safe to hand a raster in any orientation.
`RecognisesRotatedText`	Recognises rotated text correctly but emits blocks in raw raster order, so the caller must reorder.
`RequiresUpright`	Requires an upright raster; rotated text produces garbage.

ProcessingStage

Processing stages for post-processors.

Post-processors are executed in stage order (Early → Middle → Late). Use stages to control the order of post-processing operations.

Value	Description
`Early`	Early stage - foundational processing. Use for: - Language detection - Character encoding normalization - Entity extraction (NER) - Text quality scoring
`Middle`	Middle stage - content transformation. Use for: - Keyword extraction - Token reduction - Text summarization - Semantic analysis
`Late`	Late stage - final enrichment. Use for: - Custom user hooks - Analytics/logging - Final validation - Output formatting

ReductionLevel

Intensity level for the token-reduction pipeline.

Value	Description
`Off`	No reduction applied; text is returned as-is.
`Light`	Remove only the most common stopwords.
`Moderate`	Balanced stopword removal and redundancy filtering.
`Aggressive`	Aggressive filtering; may remove less common content words.
`Maximum`	Maximum compression; prioritizes brevity over completeness.

PdfAnnotationType

Type of PDF annotation.

Value	Description
`Text`	Sticky note / text annotation
`Highlight`	Highlighted text region
`Link`	Hyperlink annotation
`Stamp`	Rubber stamp annotation
`Underline`	Underline text markup
`StrikeOut`	Strikeout text markup
`Squiggly`	Squiggly (wavy) underline text markup — Since: `v1.1`
`Ink`	Freehand drawing (ink) annotation — Since: `v1.1`
`Square`	Rectangle/box shape annotation — Since: `v1.1`
`Circle`	Ellipse/oval shape annotation — Since: `v1.1`
`Polygon`	Closed polygon shape annotation — Since: `v1.1`
`PolyLine`	Open polyline shape annotation — Since: `v1.1`
`Line`	Line annotation — Since: `v1.1`
`Caret`	Caret (text-insertion marker) annotation — Since: `v1.1`
`FileAttachment`	Embedded file attachment annotation — Since: `v1.1`
`Sound`	Embedded sound annotation — Since: `v1.1`
`Movie`	Embedded movie annotation — Since: `v1.1`
`Other`	Any other annotation type

DiagramShape

Node outline, mapped onto the Graphviz shape attribute.

Value	Description
`Box`	Axis-aligned rectangle, including rounded rectangles.
`Ellipse`	Circle or ellipse.
`Diamond`	Quadrilateral standing on a vertex.
`Polygon`	Any other closed outline.

BlockType

Types of block-level elements in Djot.

Value	Description
`Paragraph`	Standard prose paragraph.
`Heading`	Section heading (level stored in `FormattedBlock::level`).
`Blockquote`	Block quotation container.
`CodeBlock`	Fenced or indented code block.
`ListItem`	Individual item within a list.
`OrderedList`	Numbered (ordered) list container.
`BulletList`	Unnumbered (bullet) list container.
`TaskList`	Task / checkbox list container.
`DefinitionList`	Definition list container.
`DefinitionTerm`	Term part of a definition list entry.
`DefinitionDescription`	Description / definition part of a definition list entry.
`Div`	Generic `div` container with optional attributes.
`Section`	Logical section container, often associated with a heading.
`ThematicBreak`	Horizontal rule / thematic break.
`RawBlock`	Raw content block in a specified format (e.g. HTML, LaTeX).
`MathDisplay`	Display-mode mathematical expression.

InlineType

Types of inline elements in Djot.

Value	Description
`Text`	Plain text run.
`Strong`	Bold / strong emphasis.
`Emphasis`	Italic / regular emphasis.
`Highlight`	Highlighted text (marker pen).
`Subscript`	Subscript text.
`Superscript`	Superscript text.
`Insert`	Inserted text (tracked change).
`Delete`	Deleted text (tracked change).
`Code`	Inline code span.
`Link`	Hyperlink with URL.
`Image`	Inline image reference.
`Span`	Generic inline span with optional attributes.
`Math`	Inline mathematical expression.
`RawInline`	Raw inline content in a specified format.
`FootnoteRef`	Footnote reference marker.
`Symbol`	Named symbol or emoji shortcode.

RelationshipKind

Semantic kind of a relationship between document elements.

Value	Description
`FootnoteReference`	Footnote marker -> footnote definition.
`CitationReference`	Citation marker -> bibliography entry.
`InternalLink`	Internal anchor link (`#id`) -> target heading/element.
`Caption`	Caption paragraph -> figure/table it describes.
`Label`	Label -> labeled element (HTML `<label for>`, LaTeX `\label{}`).
`TocEntry`	TOC entry -> target section.
`CrossReference`	Cross-reference (LaTeX `\ref{}`, DOCX cross-reference field).

ContentLayer

Content layer classification for document nodes.

Replaces separate body/furniture arrays with per-node granularity.

Value	Description
`Body`	Main document body content.
`Header`	Page/section header (running header).
`Footer`	Page/section footer (running footer).
`Footnote`	Footnote content.

NodeContent

Tagged enum for node content. Each variant carries only type-specific data.

Uses #[serde(tag = "node_type")] to avoid “type” keyword collision in Go/Java/TypeScript bindings.

Value	Description
`Title`	Document title. — Fields: `text`: `String`
`Heading`	Section heading with level (1-6). — Fields: `level`: `u8`, `text`: `String`
`Paragraph`	Body text paragraph. — Fields: `text`: `String`
`List`	List container — children are `ListItem` nodes. — Fields: `ordered`: `bool`
`ListItem`	Individual list item. — Fields: `text`: `String`
`Table`	Table with structured cell grid. — Fields: `grid`: `TableGrid`
`Image`	Image reference. — Fields: `description`: `String`, `image_index`: `u32`, `src`: `String`
`Code`	Code block. — Fields: `text`: `String`, `language`: `String`
`Quote`	Block quote — container, children carry the quoted content.
`Formula`	Mathematical formula / equation. — Fields: `text`: `String`
`Footnote`	Footnote reference content. — Fields: `text`: `String`
`Comment`	Reviewer/editor comment content (e.g. DOCX comments). Distinct from `NodeContent::Footnote` (xberg-io/xberg#300): comments and footnotes both reach the internal document via a marker/definition pair, but a consumer needs to tell a reviewer comment apart from an authored footnote. — Fields: `text`: `String` — Since: `v1.1`
`Group`	Logical grouping container (section, key-value area). `heading_level` + `heading_text` capture the section heading directly rather than relying on a first-child positional convention. — Fields: `label`: `String`, `heading_level`: `u8`, `heading_text`: `String`
`PageBreak`	Page break marker.
`Slide`	Presentation slide container — children are the slide’s content nodes. — Fields: `number`: `u32`, `title`: `String`
`DefinitionList`	Definition list container — children are `DefinitionItem` nodes.
`DefinitionItem`	Individual definition list entry with term and definition. — Fields: `term`: `String`, `definition`: `String`
`Citation`	Citation or bibliographic reference. — Fields: `key`: `String`, `text`: `String`
`Admonition`	Admonition / callout container (note, warning, tip, etc.). Children carry the admonition body content. — Fields: `kind`: `String`, `title`: `String`
`RawBlock`	Raw block preserved verbatim from the source format. Used for content that cannot be mapped to a semantic node type (e.g. JSX in MDX, raw LaTeX in markdown, embedded HTML). — Fields: `format`: `String`, `content`: `String`
`MetadataBlock`	Structured metadata block (email headers, YAML frontmatter, etc.). — Fields: `entries`: `Vec<KeyValueAttribute>`

AnnotationKind

Types of inline text annotations.

Value	Description
`Bold`	Bold (strong) text formatting.
`Italic`	Italic (emphasis) text formatting.
`Underline`	Underlined text.
`Strikethrough`	Strikethrough text.
`Code`	Inline code span.
`Subscript`	Subscript text.
`Superscript`	Superscript text.
`Link`	Hyperlink annotation. — Fields: `url`: `String`, `title`: `String`
`Highlight`	Highlighted text (PDF highlights, HTML `<mark>`).
`Color`	Text color (CSS-compatible value, e.g. “#ff0000”, “red”). — Fields: `value`: `String`
`FontSize`	Font size with units (e.g. “12pt”, “1.2em”, “16px”). — Fields: `value`: `String`
`Custom`	Extensible annotation for format-specific styling. — Fields: `name`: `String`, `value`: `String`

EntityCategory

Standard entity categories produced by built-in NER backends.

The Custom(String) variant lets caller-supplied categories (e.g. LLM schemas) flow through without losing fidelity to the consumer.

Value	Description
`Person`	A person’s name.
`Organization`	A company, institution, or organisation name.
`Location`	A geographic location (city, country, address).
`Date`	A calendar date.
`Time`	A time of day or duration.
`Money`	A monetary amount with optional currency.
`Percent`	A percentage value.
`Email`	An email address.
`Phone`	A phone number.
`Url`	A URL or URI.
`Custom`	A caller-supplied custom category label. — Fields: `0`: `String`

ExtractionMethod

How the extracted text was produced.

Value	Description
`Native`	Text extracted directly from the document’s native format (no OCR).
`Ocr`	All text was obtained via OCR (e.g. scanned image-only PDF).
`Mixed`	Text came from a combination of native extraction and OCR.

ChunkType

Semantic structural classification of a text chunk.

Assigned by the heuristic classifier in chunking::classifier. Defaults to Unknown when no rule matches. Designed to be extended in future versions without breaking changes.

Value	Description
`Heading`	Section heading or document title.
`PartyList`	Party list: names, addresses, and signatories.
`Definitions`	Definition clause (“X means…”, “X shall mean…”).
`OperativeClause`	Operative clause containing legal/contractual action verbs.
`SignatureBlock`	Signature block with signatures, names, and dates.
`Schedule`	Schedule, annex, appendix, or exhibit section.
`TableLike`	Table-like content with aligned columns or repeated patterns.
`Formula`	Mathematical formula or equation.
`CodeBlock`	Code block or preformatted content.
`Function`	Function or method definition (tree-sitter structured code chunking).
`Class`	Class, struct, interface, or trait definition (tree-sitter structured code chunking).
`Module`	Module, namespace, or top-level file scope (tree-sitter structured code chunking).
`Image`	Embedded or referenced image content.
`OrgChart`	Organizational chart or hierarchy diagram.
`Diagram`	Diagram, figure, or visual illustration.
`Unknown`	Unclassified or mixed content.

ImageKind

Heuristic classification of what an image likely depicts.

Value	Description
`Photograph`	Photographic image (natural scene, photograph)
`Diagram`	Technical or schematic diagram
`Chart`	Chart, graph, or plot
`Drawing`	Freehand or technical drawing
`TextBlock`	Text-heavy image (scanned text, document)
`Decoration`	Decorative element or border
`Logo`	Logo or brand mark
`Icon`	Small icon
`TileFragment`	Fragment of a larger tiled image (tile of a technical drawing)
`Mask`	Mask or transparency map
`PageRaster`	Full-page render produced during OCR preprocessing; used as a citation thumbnail.
`Unknown`	Could not classify with reasonable confidence

ResultFormat

Result-shape selection for extraction results.

Distinct from OutputFormat (which controls rendering — Plain, Markdown, HTML, etc.). ResultFormat controls the shape of the result: a unified content blob vs. an element-based decomposition.

Value	Description
`Unified`	Unified format with all content in `content` field
`ElementBased`	Element-based format with semantic element extraction

ElementType

Semantic element type classification.

Categorizes text content into semantic units for downstream processing. Supports the element types commonly found in Unstructured documents.

Value	Description
`Title`	Document title
`NarrativeText`	Main narrative text body
`Heading`	Section heading
`ListItem`	List item (bullet, numbered, etc.)
`Table`	Table element
`Image`	Image element
`PageBreak`	Page break marker
`CodeBlock`	Code block
`Formula`	Mathematical formula (LaTeX source in `text`)
`BlockQuote`	Block quote
`Footer`	Footer text
`Header`	Header text

FormFieldType

Kind of a PDF form field.

Mirrors xberg_native_pdf’s widget field taxonomy without leaking the upstream type across the binding surface.

Value	Description
`Text`	Single- or multi-line text input.
`Checkbox`	Checkbox (on/off toggle).
`Radio`	Radio-button group member.
`Choice`	Choice field (dropdown or list box).
`Signature`	Digital-signature field.
`Button`	Push button.
`Unknown`	Field type that could not be classified.

ElementKind

Semantic role of an internal element.

Superset of NodeContent variants plus OCR and container markers.

Value	Description
`Title`	Document title.
`Heading`	Section heading with level (1-6). — Fields: `level`: `u8`
`Paragraph`	Body text paragraph.
`ListItem`	List item. `ordered` indicates numbered vs bulleted. — Fields: `ordered`: `bool`
`Code`	Code block. Language stored in element attributes.
`Formula`	Mathematical formula / equation.
`FootnoteDefinition`	Footnote content (the definition, not the reference marker).
`FootnoteRef`	Footnote reference marker in body text.
`CommentDefinition`	Comment content (the definition, not the reference marker). See `NodeContent::Comment`.
`CommentRef`	Comment reference marker in body text.
`Citation`	Citation or bibliographic reference.
`Slide`	Presentation slide container. — Fields: `number`: `u32`
`DefinitionTerm`	Definition list term.
`DefinitionDescription`	Definition list description.
`Admonition`	Admonition / callout (note, warning, tip, etc.). Kind stored in attributes.
`RawBlock`	Raw block preserved verbatim. Format stored in attributes.
`MetadataBlock`	Structured metadata block (frontmatter, email headers).
`ListStart`	Start of a list container. — Fields: `ordered`: `bool`
`ListEnd`	End of a list container.
`QuoteStart`	Start of a block quote.
`QuoteEnd`	End of a block quote.
`GroupStart`	Start of a generic group/section.
`GroupEnd`	End of a generic group/section.
`Table`	Table reference. `table_index` is an index into `InternalDocument::tables`. — Fields: `table_index`: `u32`
`Image`	Image reference. `image_index` is an index into `InternalDocument::images`. — Fields: `image_index`: `u32`
`PageBreak`	Page break marker.
`OcrText`	OCR-detected text at a given hierarchical level. — Fields: `level`: `OcrElementLevel`

RelationshipTarget

Target of a relationship — either a resolved element index or an unresolved key.

Value	Description
`Index`	Resolved: index into `InternalDocument::elements`. — Fields: `0`: `u32`
`Key`	Unresolved: key to be matched against element anchors during derivation. — Fields: `0`: `String`

FormatMetadata

Format-specific metadata (discriminated union).

Only one format type can exist per extraction result. This provides type-safe, clean metadata without nested optionals.

Value	Description
`Pdf`	Metadata extracted from a PDF document. — Fields (flattened into the tagged object): `pdf_version`: `String`, `producer`: `String`, `is_encrypted`: `bool`, `width`: `i64`, `height`: `i64`, `page_count`: `u32`, `scanned_confidence`: `f32`, `scanned_pages`: `Vec<u32>`, `fabricated_text_pages`: `Vec<u32>`, `implausible_text_pages`: `Vec<u32>`, `layout_gated_pages`: `Vec<u32>`, `layout_gate_reasons`: `Vec<String>`
`Docx`	Metadata extracted from a DOCX Word document. — Fields (flattened into the tagged object): `core_properties`: `CoreProperties`, `app_properties`: `DocxAppProperties`, `custom_properties`: `HashMap<String, serde_json::Value>`
`Excel`	Metadata extracted from an Excel spreadsheet. — Fields (flattened into the tagged object): `sheet_count`: `u32`, `sheet_names`: `Vec<String>`
`Email`	Metadata extracted from an email message (EML/MSG). — Fields (flattened into the tagged object): `from_email`: `String`, `from_name`: `String`, `to_emails`: `Vec<String>`, `cc_emails`: `Vec<String>`, `bcc_emails`: `Vec<String>`, `message_id`: `String`, `attachments`: `Vec<String>`
`Pptx`	Metadata extracted from a PowerPoint presentation. — Fields (flattened into the tagged object): `slide_count`: `u32`, `slide_names`: `Vec<String>`, `image_count`: `u32`, `table_count`: `u32`
`Archive`	Metadata extracted from an archive (ZIP, TAR, 7Z, etc.). — Fields (flattened into the tagged object): `format`: `String`, `file_count`: `u32`, `file_list`: `Vec<String>`, `total_size`: `u64`, `compressed_size`: `u64`
`Image`	Metadata extracted from a raster or vector image. — Fields (flattened into the tagged object): `width`: `u32`, `height`: `u32`, `format`: `String`, `exif`: `HashMap<String, String>`
`Xml`	Metadata extracted from an XML document. — Fields (flattened into the tagged object): `element_count`: `u32`, `unique_elements`: `Vec<String>`
`Text`	Metadata extracted from a plain-text file. — Fields (flattened into the tagged object): `line_count`: `u32`, `word_count`: `u32`, `character_count`: `u32`, `headers`: `Vec<String>`, `links`: `Vec<MarkdownLink>`, `code_blocks`: `Vec<MarkdownCodeBlock>`
`Html`	Metadata extracted from an HTML document. — Fields (flattened into the tagged object): `title`: `String`, `description`: `String`, `keywords`: `Vec<String>`, `author`: `String`, `canonical_url`: `String`, `base_href`: `String`, `language`: `String`, `text_direction`: `TextDirection`, `open_graph`: `HashMap<String, String>`, `twitter_card`: `HashMap<String, String>`, `meta_tags`: `HashMap<String, String>`, `headers`: `Vec<HeaderMetadata>`, `links`: `Vec<LinkMetadata>`, `images`: `Vec<ImageMetadataType>`, `structured_data`: `Vec<StructuredData>`
`Ocr`	Metadata produced by an OCR pipeline. — Fields (flattened into the tagged object): `language`: `String`, `psm`: `i32`, `output_format`: `String`, `table_count`: `u32`, `table_rows`: `u32`, `table_cols`: `u32`
`Csv`	Metadata extracted from a CSV or TSV file. — Fields (flattened into the tagged object): `row_count`: `u32`, `column_count`: `u32`, `delimiter`: `String`, `has_header`: `bool`, `column_types`: `Vec<String>`
`Bibtex`	Metadata extracted from a BibTeX bibliography file. — Fields (flattened into the tagged object): `entry_count`: `usize`, `citation_keys`: `Vec<String>`, `authors`: `Vec<String>`, `year_range`: `YearRange`, `entry_types`: `HashMap<String, usize>`
`Citation`	Metadata extracted from a citation file (RIS, PubMed, EndNote). — Fields (flattened into the tagged object): `citation_count`: `usize`, `format`: `String`, `authors`: `Vec<String>`, `year_range`: `YearRange`, `dois`: `Vec<String>`, `keywords`: `Vec<String>`
`FictionBook`	Metadata extracted from a FictionBook (FB2) e-book. — Fields (flattened into the tagged object): `genres`: `Vec<String>`, `sequences`: `Vec<String>`, `annotation`: `String`
`Dbf`	Metadata extracted from a dBASE (DBF) database file. — Fields (flattened into the tagged object): `record_count`: `usize`, `field_count`: `usize`, `fields`: `Vec<DbfFieldInfo>`
`Jats`	Metadata extracted from a JATS (Journal Article Tag Suite) XML file. — Fields (flattened into the tagged object): `copyright`: `String`, `license`: `String`, `history_dates`: `HashMap<String, String>`, `contributor_roles`: `Vec<ContributorRole>`
`Epub`	Metadata extracted from an EPUB e-book. — Fields (flattened into the tagged object): `coverage`: `String`, `dc_format`: `String`, `relation`: `String`, `source`: `String`, `dc_type`: `String`, `cover_image`: `String`
`Pst`	Metadata extracted from an Outlook PST archive. — Fields (flattened into the tagged object): `message_count`: `usize`
`Audio`	Metadata extracted from an audio or video file. — Fields (flattened into the tagged object): `duration_ms`: `u64`, `codec`: `String`, `container`: `String`, `sample_rate_hz`: `u32`, `channels`: `u16`, `bitrate`: `u32`
`Code`	Code (tree-sitter analyzable source). Carries the structural chunks (function, class, and module boundaries) produced by the tree-sitter extractor, consumed by the chunking pipeline to emit structure-aware `Chunk`s instead of falling back to text-based splitting. Wraps `CodeMetadata` (a named struct) rather than `Vec<CodeChunkInfo>` directly: `FormatMetadata` is internally tagged (`#[serde(tag = "format_type")]`), and serde cannot serialize a tagged newtype variant that wraps a sequence — the tag has no map to live in. Wrapping a struct gives serde a map to hold the tag, and keeps this variant shape consistent with every sibling (`Variant(XMetadata)`) so the derived OpenAPI discriminator can reference a named component schema. — Fields (flattened into the tagged object): `chunks`: `Vec<CodeChunkInfo>`, `data`: `CodeDataNode`

CodeDataNodeKind

Discriminates the shape of a CodeDataNode.

Purpose-built mirror of tree_sitter_language_pack::DataNodeKind — kept as an xberg-owned type so binding generators never need to resolve the upstream crate’s types across FFI/language boundaries.

Value	Description
`KeyValue`	A key/value pair or mapping (JSON/TOML/properties/YAML/HCL/CUE/KDL pair, or a wrapper “object”/“mapping” container).
`Element`	An XML element with a tag name in `key` and attributes in `attributes`.
`Sequence`	A positional sequence item (JSON array element, YAML block sequence item, CSV/PSV row or cell).

TextDirection

Text direction enumeration for HTML documents.

Value	Description
`LeftToRight`	Left-to-right text direction
`RightToLeft`	Right-to-left text direction
`Auto`	Automatic text direction detection

LinkType

Link type classification.

Value	Description
`Anchor`	Anchor link (#section)
`Internal`	Internal link (same domain)
`External`	External link (different domain)
`Email`	Email link (mailto:)
`Phone`	Phone link (tel:)
`Other`	Other link type

ImageType

Image type classification.

Value	Description
`DataUri`	Data URI image
`InlineSvg`	Inline SVG
`External`	External image URL
`Relative`	Relative path image

StructuredDataType

Structured data type classification.

Value	Description
`JsonLd`	JSON-LD structured data
`Microdata`	Microdata
`Rdfa`	RDFa

OcrBoundingGeometry

Bounding geometry for an OCR element.

Supports both axis-aligned rectangles (from Tesseract) and 4-point quadrilaterals (from PaddleOCR and rotated text detection).

Value	Description
`Rectangle`	Axis-aligned bounding box (typical for Tesseract output). — Fields: `left`: `u32`, `top`: `u32`, `width`: `u32`, `height`: `u32`
`Quadrilateral`	4-point quadrilateral for rotated/skewed text (PaddleOCR). Points are in clockwise order starting from top-left: `[top_left, top_right, bottom_right, bottom_left]` — Fields: `points`: `Vec<OcrPoint>`

OcrElementLevel

Hierarchical level of an OCR element.

Maps to Tesseract’s page segmentation hierarchy and provides equivalent semantics for PaddleOCR.

Value	Description
`Word`	Individual word
`Line`	Line of text (default for PaddleOCR)
`Block`	Paragraph or text block
`Page`	Page-level element

PageUnitType

Type of paginated unit in a document.

Distinguishes between different types of “pages” (PDF pages, presentation slides, spreadsheet sheets).

Value	Description
`Page`	Standard document pages (PDF, DOCX, images)
`Slide`	Presentation slides (PPTX, ODP)
`Sheet`	Spreadsheet sheets (XLSX, ODS)

RedactionStrategy

Strategy applied when a PII match is rewritten.

Value	Description
`Mask`	Replace the matched span with a fixed mask token (default `"[REDACTED]"`).
`Hash`	Replace with a SHA-256 hash of the original value (truncated to 16 hex chars). Lets downstream consumers do equality joins without recovering the source.
`TokenReplace`	Replace with a per-category running token (`"[PERSON_1]"`, `"[PERSON_2]"`, …) so the same person referenced twice gets the same token within the document.
`Drop`	Delete the matched span entirely.

PiiCategory

PII categories the pattern engine recognises.

Value	Description
`Email`	Email address (e.g. `user@example.com`).
`Phone`	Phone number in any common format.
`Ssn`	US Social Security Number.
`CreditCard`	Payment card number (Visa, Mastercard, Amex, etc.).
`PostalCode`	Postal / ZIP code.
`IpAddress`	IPv4 or IPv6 address.
`Iban`	International Bank Account Number.
`SwiftBic`	SWIFT / BIC bank identifier code.
`DateOfBirth`	Date of birth.
`Person`	Person name, surfaced by the optional NER backend.
`Organization`	Organization name, surfaced by the optional NER backend.
`Location`	Location, surfaced by the optional NER backend.
`Custom`	Caller-supplied custom category (e.g. internal employee IDs). Surfaced by the redaction engine when a hit comes from `RedactionConfig::custom_terms` or `RedactionConfig::custom_patterns`. The string is the label passed alongside the term/pattern. Use those fields rather than constructing `Custom` directly via the `categories` filter — the pattern engine cannot detect arbitrary text from a category name alone. — Fields: `0`: `String`

RegionKind

Classification of a detected layout region that warrants VLM extraction.

Value	Description
`Figure`	A figure, diagram, chart, or image region.
`DenseTable`	A densely formatted or complex table.
`ComplexLayout`	A region with complex or mixed layout.
`Caption`	A standalone image to caption.

DiffLine

A single line in a unified-diff hunk.

Defined here (rather than only in crate::diff) so RevisionDelta can reference it unconditionally, without requiring the diff Cargo feature. crate::diff re-exports this type verbatim.

Value	Description
`Context`	Unchanged context line. — Fields: `text`: `String`
`Added`	Line added in the “after” version. — Fields: `text`: `String`
`Removed`	Line removed from the “before” version. — Fields: `text`: `String`

RevisionKind

Semantic classification of a tracked change.

Value	Description
`Insertion`	Text or content was inserted.
`Deletion`	Text or content was deleted.
`FormatChange`	Run-level formatting (font, size, colour, …) was changed.
`Comment`	A reviewer comment or annotation.

RevisionAnchor

Best-effort document location for a revision.

Value	Description
`Paragraph`	Body paragraph, identified by its zero-based index in the document flow. — Fields: `index`: `usize`
`TableCell`	Cell inside a table. — Fields: `row`: `usize`, `col`: `usize`, `table_index`: `usize`
`Page`	Page, identified by its zero-based index. — Fields: `index`: `usize`
`Slide`	Presentation slide, identified by its zero-based index. — Fields: `index`: `usize`
`Sheet`	Spreadsheet cell or range, identified by sheet index and optional name. — Fields: `index`: `usize`, `name`: `String`

SummaryStrategy

Summarisation strategy.

Value	Description
`Extractive`	Pure-Rust extractive summary (TextRank over the chunk graph). Deterministic, fast, no external service required.
`Abstractive`	Abstractive summary produced by liter-llm. Requires `liter-llm` feature and a configured `LlmConfig`. Token usage is captured in `ExtractedDocument::llm_usage`.

UriKind

Semantic classification of an extracted URI.

Value	Description
`Hyperlink`	A clickable hyperlink (web URL, file link).
`Image`	An image or media resource reference.
`Anchor`	An internal anchor or cross-reference target.
`Citation`	A citation or bibliographic reference (DOI, academic ref).
`Reference`	A general reference (e.g. `\ref{}` in LaTeX, `:ref:` in RST).
`Email`	An email address (`mailto:` link or bare email).

ExtractionSource

The source of a document to extract.

Value	Description
`File`	Extract from a filesystem path with an optional MIME type hint. — Fields: `path`: `PathBuf`, `mime_hint`: `String`
`Bytes`	Extract from in-memory bytes with a known MIME type. — Fields: `data`: `Vec<u8>`, `mime_type`: `String`

Pooling

Pooling strategy for extracting a single fixed-length vector from per-token embeddings.

Value	Description
`Cls`	Use the `[CLS]` token embedding (first token).
`Mean`	Mean of all token embeddings, weighted by attention mask.
`Last`	Use the last non-padding token embedding. Decoder-style embedding models (e.g. Qwen3-Embedding) place the pooled representation on the final content token rather than a prepended `[CLS]`.

EmbedError

Errors that can occur during text embedding inference.

Value	Description
`Tokenizer`	Tokenization failed with the given message. — Fields: `0`: `String`
`Ort`	ONNX Runtime returned an error during inference. — Fields: `0`: `String`
`Shape`	The model output tensor had an unexpected shape. — Fields: `0`: `String`
`NoOutput`	The model produced no output tensors.

EmbeddingsEmbeddingBackend

Inference backend that an EmbeddingPreset runs on.

Onnx presets require the embeddings feature (ONNX Runtime, not available on WASM/Android x86_64 emulator). Static presets require static-embeddings (pure-Rust model2vec inference, no ORT — the only dense-embedding backend available on no-ort-target).

Defaults to Onnx via #[serde(default)] so every existing preset payload (which predates this field) keeps deserializing without change.

Value	Description
`Onnx`	ONNX Runtime transformer inference (the historical, default backend).
`Static`	Pure-Rust static (model2vec) inference — no ONNX Runtime.

SparseEmbedError

Errors raised by the sparse-embedding engine.

Rust-only: the Ort variant wraps ort::Error, which has no faithful binding representation. Public callers receive crate::XbergError instead.

Value	Description
`Tokenizer`	Tokenization failed with the given message. — Fields: `0`: `String`
`Ort`	ONNX Runtime returned an error during inference. — Fields: `0`: `String`
`Shape`	The model output tensor had an unexpected shape. — Fields: `0`: `String`
`NoOutput`	The model produced no output tensors.

LateInteractionError

Errors raised by the late-interaction engine.

Rust-only: the Ort variant wraps ort::Error, which has no faithful binding representation. Public callers receive crate::XbergError instead.

Value	Description
`Tokenizer`	Tokenization failed with the given message. — Fields: `0`: `String`
`Ort`	ONNX Runtime returned an error during inference. — Fields: `0`: `String`
`Shape`	The model output tensor had an unexpected shape. — Fields: `0`: `String`
`NoOutput`	The model produced no output tensors.

KeywordAlgorithm

Keyword algorithm selection.

Value	Description
`Yake`	YAKE (Yet Another Keyword Extractor) - statistical approach
`Rake`	RAKE (Rapid Automatic Keyword Extraction) - co-occurrence based

SchemaCompliance

Schema-validation outcome surfaced as one of three buckets.

Fold into the combined confidence score without leaking internal validation error types.

Value	Description
`AllValid`	Every batch validated against the schema.
`PartialValid`	At least one batch validated; at least one did not.
`AllInvalid`	No batch validated.

NoChunkingReason

Reason for not chunking a document.

Value	Description
`SmallFile`	File is below size threshold. — Fields: `size_bytes`: `u64`, `threshold_bytes`: `u64`
`FewPages`	Document has fewer pages than threshold. — Fields: `page_count`: `u32`, `threshold`: `u32`
`TextLayerDetected`	PDF has substantial text layer (OCR not needed). — Fields: `text_coverage`: `f32`, `avg_chars_per_page`: `u32`
`FormatNotChunkable`	Document format does not support chunking. — Fields: `mime_type`: `String`
`ChunkingDisabled`	Chunking is disabled by configuration.
`FastTextExtraction`	Force OCR is disabled and text extraction is fast.

ChunkingReason

Reason for chunking a document.

Value	Description
`LargeFile`	File exceeds size threshold. — Fields: `size_bytes`: `u64`, `threshold_bytes`: `u64`
`ManyPages`	Document has many pages. — Fields: `page_count`: `u32`, `threshold`: `u32`
`OcrRequired`	PDF requires OCR and is large. — Fields: `page_count`: `u32`, `force_ocr`: `bool`
`LargeAndManyPages`	Both size and page count exceed thresholds. — Fields: `size_bytes`: `u64`, `page_count`: `u32`

BoundaryReason

Reason for boundary detection.

Value	Description
`Start`	Start of PDF.
`PageOneMarker`	Page-one marker (“Page 1”, “1 of N”) detected.
`LetterheadReset`	Letterhead reset after signature block.
`DensityShift`	Text density shift with low bigram overlap.
`End`	End of PDF.

PresetCategory

High-level category used to group presets in the registry UI.

Value	Description
`Finance`	Invoices, receipts, statements, purchase orders, W-9.
`Identity`	Passports, drivers licenses, insurance cards.
`Legal`	Contracts, NDAs, agreements.
`Logistics`	Bills of lading, customs declarations, packing lists.
`Medical`	Clinical records, lab reports.
`Hr`	Pay stubs, resumes, employment offers.
`Other`	Catch-all for documents that don’t fit the other categories.

OcrError

OCR-specific errors (pure Rust, no PyO3).

Value	Description
`TesseractInitializationFailed`	Tesseract failed to initialize with the given configuration. — Fields: `0`: `String`
`UnsupportedVersion`	Tesseract version is below the minimum supported version. — Fields: `0`: `String`
`InvalidConfiguration`	Configuration parameter is invalid or out of range. — Fields: `0`: `String`
`InvalidLanguageCode`	Language code is not recognized by Tesseract. — Fields: `0`: `String`
`ImageProcessingFailed`	Image preprocessing or decoding failed. — Fields: `0`: `String`
`ProcessingFailed`	The OCR recognition step itself failed. — Fields: `0`: `String`
`CacheError`	Reading or writing the OCR result cache failed. — Fields: `0`: `String`
`IoError`	An I/O error occurred while reading an image or writing output. — Fields: `0`: `String`

PSMMode

Page Segmentation Mode for Tesseract OCR.

Value	Description
`OsdOnly`	Orientation and script detection only.
`AutoOsd`	Automatic page segmentation with OSD.
`AutoOnly`	Automatic page segmentation without OSD or OCR.
`Auto`	Fully automatic page segmentation with no OSD (default).
`SingleColumn`	Assume a single column of text of variable sizes.
`SingleBlockVertical`	Assume a single uniform block of vertically aligned text.
`SingleBlock`	Assume a single uniform block of text.
`SingleLine`	Treat the image as a single text line.
`SingleWord`	Treat the image as a single word.
`CircleWord`	Treat the image as a single word in a circle.
`SingleChar`	Treat the image as a single character.

ProbeStatus

Outcome of a single doctor check.

Value	Description
`Pass`	The backend or setting will work as configured.
`Warn`	The check ran and found something actionable, but nothing is broken (e.g. stray cache files, stale model revisions). Never fails the report.
`Fail`	The configured setup will not work (or will silently degrade) on this host.
`Skip`	The check cannot run locally (e.g. model not cached, feature not compiled in); first real use decides, possibly after a download.

PaddleInferenceBackend

Which concrete ONNX inference engine PaddleOCR model loading uses.

Mirrors sceptre::Backend for the PaddleOCR backend: Ort is the native, full-featured path (acceleration/execution-provider hook, ONNX-embedded dictionary metadata); Tract is the pure-Rust, CPU-only path used on targets where ort cannot link (Android x86_64 emulator, WASM once wired).

Value	Description
`Ort`	Native ONNX Runtime (requires the `paddle-ocr-ort` feature).
`Tract`	Pure-Rust ONNX via `tract` (requires the `paddle-ocr-tract` feature).

PaddleLanguage

Supported languages in PaddleOCR.

Maps user-friendly language codes to paddle-ocr-rs language identifiers.

Value	Description
`English`	English
`Chinese`	Simplified Chinese
`Japanese`	Japanese
`Korean`	Korean
`German`	German
`French`	French
`Latin`	Latin script (covers most European languages)
`Cyrillic`	Cyrillic (Russian and related)
`TraditionalChinese`	Traditional Chinese
`Thai`	Thai
`Greek`	Greek
`EastSlavic`	East Slavic (Russian, Ukrainian, Belarusian)
`Arabic`	Arabic (Arabic, Persian, Urdu)
`Devanagari`	Devanagari (Hindi, Marathi, Sanskrit, Nepali)
`Tamil`	Tamil
`Telugu`	Telugu

CandleDevicePreference

Device selection shared by the typed candle backend option objects.

Value	Description
`Auto`	Auto
`Cpu`	Cpu
`Cuda`	Cuda
`Metal`	Metal

CandleTrocrVariant

TrOCR model variant accepted by candle-trocr backend options.

Value	Description
`BasePrinted`	Base printed
`LargePrinted`	Large printed
`BaseHandwritten`	Base handwritten
`LargeHandwritten`	Large handwritten

PaddleOcrVlTaskKind

Task accepted by candle-paddleocr-vl backend options.

Value	Description
`Ocr`	Ocr
`Table`	Table element
`Formula`	Formula
`Chart`	Chart

GlmOcrTaskKind

Task accepted by candle-glm-ocr backend options.

Value	Description
`Ocr`	Ocr
`Table`	Table element
`Formula`	Formula
`Chart`	Chart
`Caption`	Caption element

GlmOcrLayoutMode

Page layout mode accepted by candle-glm-ocr backend options.

Value	Description
`WholePage`	Whole page
`Paired`	Paired

CandleDeepseekOcrDtype

Floating-point precision accepted by candle-deepseek-ocr backend options.

Auto (the default) resolves per compute device: BF16 on CUDA, F16 on Metal, F32 on CPU. A dtype with no kernel on the selected device fails the load hard rather than silently falling back to another precision – this is not a tuning knob.

Value	Description
`Auto`	Auto
`F32`	F32
`F16`	F16
`Bf16`	Bf16

CandleModelId

Identifier used by CandleOcrConfig::backend_name for its legacy model pair.

Value	Description
`Trocr`	Trocr
`PaddleocrVl`	Paddleocr vl

LayoutMode

How the backend dispatches inference across a page image.

WholePage passes the raw page bytes to the engine as a single call — fast and simple. Paired (compiled only when layout-detection is enabled) runs PP-DocLayout-V3 first, crops individual regions, and dispatches each crop to the task that best matches the detected layout class, merging results in reading order.

Value	Description
`WholePage`	Feed the whole page to the model with the backend’s default task.
`Paired`	Detect layout regions first; dispatch each crop to the matching task. Only available when the `layout-detection` feature is enabled.

LayoutClass

The 18 canonical document layout classes.

All model backends (RT-DETR, YOLO, etc.) map their native class IDs to this shared set. Models with fewer classes (DocLayNet: 11, PubLayNet: 5) map to the closest equivalent.

Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
`Caption`	Figure or table caption text.
`Chart`	Chart or graph visualization.
`Footnote`	Footnote or endnote text.
`Formula`	Mathematical formula or equation.
`ListItem`	A single item in a bulleted or numbered list.
`PageFooter`	Running footer at the bottom of a page.
`PageHeader`	Running header at the top of a page.
`Picture`	Image, chart, or other graphical element.
`SectionHeader`	Section heading.
`Table`	Data table.
`Text`	Body text paragraph.
`Title`	Document or chapter title.
`DocumentIndex`	Table of contents or index.
`Code`	Source code block.
`CheckboxSelected`	Checkbox in selected state.
`CheckboxUnselected`	Checkbox in unselected state.
`Form`	Form field or form element.
`KeyValueRegion`	Key-value pair region (e.g. label + value in a form).

ModelBackend

Which underlying model architecture to use for layout detection.

Value	Description
`YoloDocLayNet`	YOLO trained on DocLayNet (11 classes, 640x640 input).
`RtDetr`	RT-DETR v2 (17 classes, 640x640 input, NMS-free).
`PpDocLayoutV3`	PP-DocLayout-V3 (25 classes, 800×800 input, PaddleDetection DETR).
`Custom`	Custom model from a local file path. — Fields: `path`: `PathBuf`, `variant`: `CustomModelVariant`

CustomModelVariant

Variant selection for custom model paths, used with ModelBackend::Custom.

Value	Description
`RtDetr`	RT-DETR v2 model format.
`PpDocLayoutV3`	PP-DocLayout-V3 model format.
`YoloDocLayNet`	YOLO trained on DocLayNet (11 classes).
`YoloDocStructBench`	DocLayout-YOLO trained on DocStructBench (10 classes).
`Yolox`	YOLOX with explicit input dimensions. — Fields: `input_width`: `u32`, `input_height`: `u32`

TableType

Table type classification result.

Value	Description
`Wired`	Bordered table with visible gridlines.
`Wireless`	Borderless table without visible gridlines.

TatrClass

TATR object detection class labels.

The 7 classes output by the Table Transformer model. NoObject (class 6) is the background/padding class and is filtered out during post-processing.

Value	Description
`Table`	Full table bounding box (class 0).
`Column`	Table column (class 1).
`Row`	Table row (class 2).
`ColumnHeader`	Column header row (class 3).
`ProjectedRowHeader`	Projected row header column (class 4).
`SpanningCell`	Spanning cell covering multiple rows/columns (class 5).

YoloVariant

Which YOLO variant this model represents.

Value	Description
`DocLayNet`	YOLOv10/v8 trained on DocLayNet (11 classes). Output: [batch, num_dets, 6] = [x1, y1, x2, y2, score, class_id]
`DocStructBench`	DocLayout-YOLO trained on DocStructBench (10 classes). Output: [batch, num_dets, 4+num_classes] center-format, or [batch, num_dets, 6] decoded.
`Yolox`	YOLOX with letterbox preprocessing and grid decoding. Output: [batch, num_anchors, 5+num_classes] — needs grid decoding + NMS. Strides: [8, 16, 32], anchors decoded via (raw + grid_offset) * stride.

PdfError

PDF-specific errors produced by the pure-Rust PDF backend.

Value	Description
`InvalidPdf`	The file is not a valid PDF or the header is corrupted. — Fields: `0`: `String`
`PasswordRequired`	The PDF is encrypted and no password was provided.
`InvalidPassword`	The provided password did not decrypt the document.
`EncryptionNotSupported`	The encryption scheme is not supported by the pure-Rust backend. — Fields: `0`: `String`
`PageNotFound`	The requested page index does not exist in the document. — Fields: `0`: `usize`
`TextExtractionFailed`	Extracting the text layer from a page failed. — Fields: `0`: `String`
`RenderingFailed`	Page rendering to an image failed. — Fields: `0`: `String`
`MetadataExtractionFailed`	Reading or parsing the PDF document info dictionary failed. — Fields: `0`: `String`
`ExtractionFailed`	A general extraction failure not covered by a more specific variant. — Fields: `0`: `String`
`FontLoadingFailed`	A required font could not be loaded or decoded. — Fields: `0`: `String`
`IoError`	An I/O error occurred while reading the PDF stream. — Fields: `0`: `String`
`Cancelled`	The operation was cancelled (reachable only via the REST async-jobs API).

AuthConfig

Authentication configuration.

Value	Description
`Basic`	HTTP Basic authentication. — Fields: `username`: `String`, `password`: `String`
`Bearer`	Bearer token authentication. — Fields: `token`: `String`
`Header`	Custom authentication header. — Fields: `name`: `String`, `value`: `String`

BrowserMode

When to use the headless browser fallback.

Value	Description
`Auto`	Automatically detect when JS rendering is needed and fall back to browser.
`Always`	Always use the browser for every request.
`Never`	Never use the browser fallback.
`Stealth`	Always use the browser with all stealth surfaces enabled. Behaves like `Always` for escalation purposes (every request is routed through the browser tier), but additionally enables: - browser JavaScript stealth patches - native-backend TLS fingerprint spoofing - stealth-aware default user-agent when no explicit UA is set - 1920×1080 viewport override Use this instead of setting the now-removed `BrowserConfig.stealth` boolean field.

BrowserWait

Wait strategy for browser page rendering.

Value	Description
`NetworkIdle`	Wait until network activity is idle.
`Selector`	Wait for a specific CSS selector to appear in the DOM.
`Fixed`	Wait for a fixed duration after navigation.

BrowserBackend

Browser backend used for JavaScript rendering.

Value	Description
`Chromiumoxide`	Existing Chromium/CDP backend powered by chromiumoxide.
`Native`	Crawlberg-owned native browser backend derived from Obscura.

DocumentContentEncoding

Opt-in encoding applied to a downloaded document’s bytes for callers who need the content available in a serializable field rather than reading it from disk.

None (the CrawlConfig.document_content_encoding default) produces neither — unlike screenshots, base64-encoding a document by default would duplicate an already up-to-document_max_size buffer (50 MB default) in memory per document.

Value	Description
`Base64`	Populate `DownloadedDocument.content_base64` with a base64-encoded copy.

CrawlStrategyKind

Traversal order for a crawl.

Selects both the queue discipline and the selection strategy, because global order is a property of the frontier: the engine hands its bounded selection window to the strategy, so a strategy alone can only reorder URLs that have already been dequeued.

Value	Description
`Bfs`	Breadth-first: a FIFO frontier visits every URL at one depth before the next.
`Dfs`	Depth-first: a LIFO frontier descends into a page’s children before its siblings.
`BestFirst`	Highest-priority-first within the selection window, scored by `CrawlStrategy::score_url`.
`Adaptive`	Like `BestFirst`, but stops once newly crawled pages stop contributing new terms.

ContentFilterKind

Content filter applied to each crawled page before it reaches the result.

Value	Description
`Bm25`	Keep only pages scoring at or above `bm25_threshold` for `bm25_query`.

AssetCategory

The category of a downloaded asset.

Value	Description
`Document`	A document file (PDF, DOC, etc.).
`Image`	An image file.
`Audio`	An audio file.
`Video`	A video file.
`Font`	A font file.
`Stylesheet`	A CSS stylesheet.
`Script`	A JavaScript file.
`Archive`	An archive file (ZIP, TAR, etc.).
`Data`	A data file (JSON, XML, CSV, etc.).
`Other`	An unrecognized asset type.

HostMatcher

Hostname/IP allowlist matcher for SSRF policy.

Serializes as an internally-tagged object so each variant is distinguishable on the wire and round-trips losslessly:

{"type": "exact",  "value": "api.example.com"}
{"type": "suffix", "value": ".example.com"}
{"type": "cidr",   "value": "10.0.0.0/8"}

A bare JSON string is still accepted on deserialization and resolves to Exact, preserving configs written against the previous untagged representation.

Exact: HostMatcher::Exact

Value	Description
`Exact`	Exact hostname match (case-insensitive). — Fields: `value`: `String`
`Suffix`	Suffix match: “.xberg.io” matches “api.xberg.io” and “xberg.io”. — Fields: `value`: `String`
`Cidr`	CIDR match: “10.0.0.0/8” matches IP addresses in that range. — Fields: `value`: `String`

TierStrategy

Controls which conversion tier is used.

Value	Description
`Auto`	Automatically pick the best tier for the input (default). Runs the classifier against the prescan report and uses Tier-1 when eligible; falls back to Tier-2 on bail or when the classifier routes to Tier-2.
`Tier2`	Always use the Tier-2 (`tl::parse` + walk) path, skipping Tier-1.

PreprocessingPreset

HTML preprocessing aggressiveness level.

Controls the extent of cleanup performed before conversion. Higher levels remove more elements.

Value	Description
`Minimal`	Minimal cleanup. Remove only essential noise (scripts, styles).
`Standard`	Standard cleanup. Default. Removes navigation, forms, and other auxiliary content.
`Aggressive`	Aggressive cleanup. Remove extensive non-content elements and structure.

HeadingStyle

Heading style options for Markdown output.

Controls how headings (h1-h6) are rendered in the output Markdown.

Value	Description
`Underlined`	Underlined style (=== for h1, — for h2).
`Atx`	ATX style (# for h1, ## for h2, etc.). Default.
`AtxClosed`	ATX closed style (# title #, with closing hashes).

ListIndentType

List indentation character type.

Controls whether list items are indented with spaces or tabs.

Value	Description
`Spaces`	Use spaces for indentation. Default. Width controlled by `list_indent_width`.
`Tabs`	Use tabs for indentation.

WhitespaceMode

Whitespace handling strategy during conversion.

Determines how sequences of whitespace characters (spaces, tabs, newlines) are processed.

Value	Description
`Normalized`	Collapse multiple whitespace characters to single spaces. Default. Matches browser behavior.
`Strict`	Preserve all whitespace exactly as it appears in the HTML.

NewlineStyle

Line break syntax in Markdown output.

Controls how soft line breaks (from <br> or line breaks in source) are rendered.

Value	Description
`Spaces`	Two trailing spaces at end of line. Default. Standard Markdown syntax.
`Backslash`	Backslash at end of line. Alternative Markdown syntax.

CodeBlockStyle

Code block fence style in Markdown output.

Determines how code blocks (<pre><code>) are rendered in Markdown.

Value	Description
`Indented`	Indented code blocks (4 spaces). `CommonMark` standard.
`Backticks`	Fenced code blocks with triple backticks. Default (GFM). Supports language hints.
`Tildes`	Fenced code blocks with tildes (~~~). Supports language hints.

HighlightStyle

Highlight rendering style for <mark> elements.

Controls how highlighted text is rendered in Markdown output.

Value	Description
`DoubleEqual`	Double equals syntax (==text==). Default. Pandoc-compatible.
`Html`	Preserve as HTML (==text==). Original HTML tag.
`Bold`	Render as bold (text). Uses strong emphasis.
`None`	Strip formatting, render as plain text. No markup.

LinkStyle

Link rendering style in Markdown output.

Controls whether links and images use inline text syntax or reference-style [text][1] syntax with definitions collected at the end.

Value	Description
`Inline`	Inline links: `text`. Default.
`Reference`	Reference-style links: `[text][1]` with `[1]: url` at end of document.

UrlEscapeStyle

URL encoding strategy for link and image destinations.

Controls how special characters in URL destinations are handled when they require escaping to produce valid Markdown.

The Angle variant (default) wraps the destination in angle brackets: [text](<url with spaces>). This is the CommonMark-specified escape hatch but breaks when the URL itself contains >.

The Percent variant percent-encodes every character that is not an RFC 3986 unreserved character or /, producing a destination safe for all Markdown parsers: [text](url%20with%20spaces).

Value	Description
`Angle`	Wrap destinations that contain spaces or newlines in angle brackets. Default.
`Percent`	Percent-encode all characters that are not RFC 3986 unreserved or `/`.

InlineDataMedia

What the output shows for an image, an embedded media element, or a link, whose address is an inline data: URL.

Applies to <img> (including its lazy-load attributes, its srcset candidates and the <source> elements of a <picture> around it), <graphic>, inline <svg>, <video>, <audio> (including their nested <source> elements), <iframe>, and a link (<a href>) whose own address is a data: URL.

With AltTextOnly or DropElement, an element that also has an address that is not data: uses that address instead. The document structure follows the markdown: a dropped image has no node, and an image written as its alt text has no address. A link whose only content the choice removed is dropped with it.

A link has no attribute separate from its own text to use as a caption, so AltTextOnly and DropElement do the same thing to a link’s data: address: write the link’s text with no destination. Unlike an image, a link’s text is never dropped along with the address.

Value	Description
`Keep`	Write the `data:` URL as the destination, payload included. Default.
`AltTextOnly`	Write the alt text (the title for an `<svg>`, the fallback content for `<video>` and `<audio>`) without a destination.
`DropElement`	Write nothing for the element.

HtmlToMarkdownRsOutputFormat

Output format for conversion.

Specifies the target markup language format for the conversion output.

Value	Description
`Markdown`	Standard Markdown (`CommonMark` compatible). Default.
`Djot`	Djot lightweight markup language.
`Plain`	Plain text output (no markup, visible text only).

Errors

XbergError

Main error type for all Xberg operations.

All errors in Xberg use this enum, which preserves error chains and provides context for debugging.

Variants

Io - File system and I/O errors (always bubble up)
Parsing - Document parsing errors (corrupt files, unsupported features)
Ocr - OCR processing errors
Validation - Input validation errors (invalid paths, config, parameters)
Cache - Cache operation errors (non-fatal, can be ignored)
ImageProcessing - Image manipulation errors
Serialization - JSON/MessagePack serialization errors
MissingDependency - Missing optional dependencies (tesseract, etc.)
Plugin - Plugin-specific errors
LockPoisoned - Mutex/RwLock poisoning (should not happen in normal operation)
UnsupportedFormat - Unsupported MIME type or file format
Other - Catch-all for uncommon errors

FFI error codes — a stable public contract

Each variant carries #[cfg_attr(alef, alef(error_code = N))]. Without it alef has no stable numeric taxonomy to bind to, so alef_ffi_error_code() collapses every variant to a single unknown code and EVERY language binding loses the ability to tell error kinds apart – errors.Is(err, ErrOcr) in Go, the checkLastError switch in Java, and Zig’s typed error sets all degrade to one catch-all. Refusing to guess is correct on alef’s part; supplying the codes is our job.

The blocks are XbergError 1000-1017, HeuristicsError 1100-1101, LoadError 1200-1205, ResolveError 1300, chosen to leave room to grow and to stay clear of alef’s own reserved 0-4 (None/Conversion/Unknown/Panic/InvalidHandle).

These numbers cross the FFI boundary and are compiled into released bindings, so they are append-only: never renumber or reuse a code, and give a new variant the next free number in its block rather than inserting one in declaration order.

Variant	Description
`Io`	A file system or I/O operation failed. These errors always bubble up unchanged.
`Parsing`	Document parsing failed (e.g. corrupt file, unsupported format feature).
`Ocr`	An OCR engine returned an error or produced unusable output.
`Validation`	Invalid configuration or input parameters were supplied.
`Cache`	A cache read or write operation failed.
`ImageProcessing`	An image manipulation operation (resize, decode, DPI conversion) failed.
`Serialization`	JSON or MessagePack serialization/deserialization failed.
`MissingDependency`	A required optional system dependency (e.g. `tesseract`) was not found.
`Plugin`	A registered plugin returned an error during extraction.
`LockPoisoned`	An internal `Mutex` or `RwLock` was found in a poisoned state.
`UnsupportedFormat`	The document’s MIME type is not supported by any registered extractor.
`Embedding`	The embedding model or embedding pipeline returned an error.
`Reranking`	The reranker model or reranking pipeline returned an error.
`Transcription`	Audio/video transcription failed.
`Timeout`	The extraction operation exceeded the configured time limit.
`Cancelled`	The extraction stopped after a cooperative cancellation request. Rust callers can request cancellation through `CancellationToken`. The REST async-jobs API uses the same mechanism. Generated language bindings do not currently expose in-process cancellation.
`Security`	A security policy was violated (e.g. zip bomb, oversized archive).
`Other`	A catch-all for uncommon errors that do not fit another variant.

RerankError

Errors that can occur during cross-encoder reranking inference.

Variant	Description
`Tokenizer`	Tokenization failed with the given message.
`Ort`	ONNX Runtime returned an error during inference.
`Shape`	The model output tensor had an unexpected shape.
`NoOutput`	The model produced no output tensors.

HeuristicsError

Errors that can occur during heuristics analysis.

Variant	Description
`ConfigError`	Invalid configuration value.
`PdfAnalysisError`	PDF analysis step failed (only when `heuristics-pdf` feature is active).

LoadError

Errors produced while loading or validating a preset file.

Variant	Description
`Parse`	The file is not valid JSON.
`SchemaValidation`	The file parses as JSON but does not validate against the meta-schema.
`Deserialize`	The file validates but cannot be deserialized into `Preset`.
`IdMismatch`	The preset’s declared `id` does not match its file-system location.
`BadMetaSchema`	The meta-schema itself failed to compile.
`Io`	A filesystem I/O error occurred while reading a preset directory.

ResolveError

Errors produced while resolving a preset against caller overrides.

Variant	Description
`SchemaNotObject`	A custom schema override was supplied but is not a JSON object.

LayoutError

Errors that can occur during layout detection inference or model management.

Variant	Description
`Inference`	A model on the engine-neutral `inference` seam failed to load or run. Engine-agnostic (ORT or tract) — carries the seam error’s message so the layout models need not name a concrete engine’s error type.
`Image`	Image decoding or preprocessing failed.
`SessionNotInitialized`	The ONNX session was not initialized before calling `detect`.
`InvalidOutput`	The model returned output tensors with an unexpected shape or type.
`ModelDownload`	Downloading or locating the model weights file failed.

TranscriptionError

Errors that can occur during Whisper inference.

Variant	Description
`Ort`	ONNX Runtime returned an error during session build or inference.
`Tokenizer`	Tokenizer load or decode failed.
`Shape`	A tensor shape was not as expected.
`Io`	An I/O error occurred (model file missing, etc.).
`NoOutput`	The encoder produced no output.
`MissingSpecialToken`	A required special token was missing from the tokenizer vocabulary.
`MelSpec`	The mel spectrogram computation failed.

WhisperModelError

Errors that can occur while resolving Whisper model paths.

Variant	Description
`ModelMissing`	The model is not cached locally and `allow_network` is `false`.
`Download`	A file download from Hugging Face Hub failed.
`Io`	An I/O error occurred while loading a resolved model artifact.
`Cache`	The resolved cache directory could not be determined or created.
`HashVerificationUnavailable`	Hash verification was requested, but the resolver has no checked-in checksum manifest for the pinned Whisper snapshots.

Rust API Reference