TypeScript API Reference

TypeScript API Reference v1.1.0

Functions

extract()

Extract content from a single bytes or URI input.

Signature:

function extract(input: ExtractInput, config: ExtractionConfig): Promise<ExtractionResult>

Example:

const result = await extract(new ExtractInput(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
input	ExtractInput	Yes	The input data
config	ExtractionConfig	Yes	The configuration options

Returns: ExtractionResult

Errors: Throws Error with a descriptive message.

---

extractBatch()

Extract content from multiple bytes or URI inputs.

Signature:

function extractBatch(inputs: Array<ExtractInput>, config: ExtractionConfig): Promise<ExtractionResult>

Example:

const result = await extractBatch([], new ExtractionConfig());

Parameters:

Name	Type	Required	Description
inputs	Array<ExtractInput>	Yes	The inputs
config	ExtractionConfig	Yes	The configuration options

Returns: ExtractionResult

Errors: Throws Error with a descriptive message.

---

mapUrl()

Discover all pages and sitemaps reachable from uri without extracting document content.

Builds a crawlberg.CrawlEngine from config.crawl, calls CrawlEngine.map, and returns the set of discovered URLs as a crawlberg.MapResult (re-exported as MapResult).

Use this when you need the URL inventory of a site before committing to full document extraction — e.g. to build a crawl queue or validate scope.

Errors:

Returns Validation if the crawl configuration fails validation or if the map operation itself fails.

Signature:

function mapUrl(uri: string, config: UrlExtractionConfig): Promise<MapResult>

Example:

const result = await mapUrl("value", new UrlExtractionConfig());

Parameters:

Name	Type	Required	Description
uri	string	Yes	The uri
config	UrlExtractionConfig	Yes	The configuration options

Returns: MapResult

Errors: Throws Error with a descriptive message.

---

listSupportedFormats()

List all supported document formats.

Returns every file extension Xberg recognizes together with its corresponding MIME type, derived from the central format registry. Formats that have no registered file extension (such as source code, which is detected dynamically) are not included.

The static EXT_TO_MIME table lists every format the codebase knows how to describe, regardless of which Cargo features were compiled in. Advertising that table directly would claim support for extractors that may not exist in this build (see GH#1387). To keep the advertised catalogue honest, the table is intersected with the document extractor registry: an extension is only included if some registered extractor actually claims its MIME type in this build. This can never drift from reality and automatically covers third-party extractors registered at runtime.

The list is sorted alphabetically by file extension.

Returns:

A vector of SupportedFormat entries sorted by extension, limited to formats with a registered extractor in this build.

Signature:

function listSupportedFormats(): Array<SupportedFormat>

Example:

const result = listSupportedFormats();

Returns: Array<SupportedFormat>

---

ensureInitialized()

Ensure built-in extractors are registered.

This function is called automatically on first extraction operation. It’s safe to call multiple times - registration only happens once, unless the registry was cleared, in which case extractors are re-registered.

Public so a caller that wants to inspect the registry — rather than extract — can populate it directly. Without this the only way to trigger registration is to run a real extraction, which xberg formats would otherwise have to fake (#233).

Signature:

function ensureInitialized(): void

Example:

ensureInitialized();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

clearEmbeddingBackends()

Clear all embedding backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

• Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

function clearEmbeddingBackends(): void

Example:

clearEmbeddingBackends();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listEmbeddingBackends()

List the names of all registered embedding backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

function listEmbeddingBackends(): Array<string>

Example:

const result = listEmbeddingBackends();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

registerDocumentExtractor()

Register a document extractor with the global registry.

The extractor is keyed by name and indexed for every MIME type returned by supported_mime_types.

Errors:

• Validation if the plugin name is empty or contains whitespace.

• Any error returned by the extractor’s initialize() method.

Signature:

function registerDocumentExtractor(extractor: DocumentExtractor): void

Example:

registerDocumentExtractor(new DocumentExtractor());

Parameters:

Name	Type	Required	Description
extractor	DocumentExtractor	Yes	The document extractor

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

unregisterDocumentExtractor()

Unregister a document extractor by name.

Removes the extractor from the global registry and calls its shutdown() method. No-op if no extractor with that name is registered.

Errors:

• Any error returned by the extractor’s shutdown() method.

Signature:

function unregisterDocumentExtractor(name: string): void

Example:

unregisterDocumentExtractor("value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listDocumentExtractors()

List names of all registered document extractors.

Signature:

function listDocumentExtractors(): Array<string>

Example:

const result = listDocumentExtractors();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearDocumentExtractors()

Clear all document extractors from the global registry.

Calls shutdown() on every registered extractor, then empties the registry.

Errors:

• Any error returned by an extractor’s shutdown() method. The first error encountered stops processing of remaining extractors.

Signature:

function clearDocumentExtractors(): void

Example:

clearDocumentExtractors();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listOcrBackends()

List all registered OCR backends.

Returns the names of all OCR backends currently registered in the global registry.

Returns:

A vector of OCR backend names.

Signature:

function listOcrBackends(): Array<string>

Example:

const result = listOcrBackends();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearOcrBackends()

Clear all OCR backends from the global registry.

Removes all OCR backends and calls their shutdown() methods.

Returns:

• Ok(()) if all backends were cleared successfully
• Err(...) if any shutdown method failed

Signature:

function clearOcrBackends(): void

Example:

clearOcrBackends();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listPostProcessors()

List all registered post-processor names.

Returns a vector of all post-processor names currently registered in the global registry.

Returns:

• Ok(string[]) - Vector of post-processor names
• Err(...) if the registry lock is poisoned

Signature:

function listPostProcessors(): Array<string>

Example:

const result = listPostProcessors();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearPostProcessors()

Remove all registered post-processors.

Signature:

function clearPostProcessors(): void

Example:

clearPostProcessors();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

registerRenderer()

Register a renderer plugin with the global registry.

The renderer’s format name is taken from Plugin.name. Registering a renderer with a name that already exists replaces the previous renderer for that format.

Note on Result return type

Returns Result<()> for cross-language API symmetry required by the alef trait-bridge codegen. The underlying parking_lot.RwLock cannot be poisoned (parking_lot provides no poisoning semantics), so this function never returns Err in practice.

Signature:

function registerRenderer(renderer: Renderer): void

Example:

registerRenderer(new Renderer());

Parameters:

Name	Type	Required	Description
renderer	Renderer	Yes	The renderer

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

unregisterRenderer()

Unregister a renderer by format name.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

function unregisterRenderer(name: string): void

Example:

unregisterRenderer("value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listRenderers()

List names of all registered renderers.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

function listRenderers(): Array<string>

Example:

const result = listRenderers();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearRenderers()

Clear all renderers from the global registry.

Removes every renderer, including the built-in defaults (markdown, html, djot, plain). After calling this no renderers are registered; re-register as needed.

Errors:

Returns an error if the registry lock is poisoned.

Signature:

function clearRenderers(): void

Example:

clearRenderers();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

clearRerankerBackends()

Clear all reranker backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

• Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Since v5.0.

Signature:

function clearRerankerBackends(): void

Example:

clearRerankerBackends();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listRerankerBackends()

List the names of all registered reranker backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Since v5.0.

Signature:

function listRerankerBackends(): Array<string>

Example:

const result = listRerankerBackends();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearTokenizerBackends()

Clear all tokenizer backends from the global registry.

Calls shutdown() on every registered backend, then empties the registry.

Errors:

• Any error returned by a backend’s shutdown() method. The first error encountered stops processing of remaining backends.

Signature:

function clearTokenizerBackends(): void

Example:

clearTokenizerBackends();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

listTokenizerBackends()

List the names of all registered tokenizer backends.

Used by xberg-cli, the api/mcp endpoints, and generated language bindings.

Signature:

function listTokenizerBackends(): Array<string>

Example:

const result = listTokenizerBackends();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

listValidators()

List names of all registered validators.

Signature:

function listValidators(): Array<string>

Example:

const result = listValidators();

Returns: Array<string>

Errors: Throws Error with a descriptive message.

---

clearValidators()

Remove all registered validators.

Signature:

function clearValidators(): void

Example:

clearValidators();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

classifyChunks()

Run chunk classification against an extraction result.

Mutates ChunkMetadata.classifications on every chunk in result.chunks and appends every LLM call’s usage to result.llm_usage. A chunk whose classification batch call fails (or that the model omitted from its response) is simply left with an empty classifications vector for that chunk, unless the failure is a validation error (empty config) or every batch task fails, in which case the first error is returned.

Errors:

Returns Validation when config.definitions is empty. Returns the first batch error encountered when rendering the prompt or calling the LLM fails for every batch; partial failures on a subset of batches are recorded here as a ProcessingWarning on result instead of aborting the whole run.

Signature:

function classifyChunks(result: ExtractedDocument, config: ChunkClassificationConfig): Promise<void>

Example:

await classifyChunks(new ExtractedDocument(), new ChunkClassificationConfig());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document
config	ChunkClassificationConfig	Yes	The configuration options

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

findUnmarkedClaims()

Find unmarked claims in markdown text.

Returns lines that assert a claim but carry neither a footnote citation anchor ([^...]) nor an inference marker ([*inference*]).

The heuristic is simple: a line that contains alphabetic words, ends with sentence punctuation, and is not a heading, blank line, or markup-only line is considered a claim. Exclude lines that appear in the citation block (after --- + <!-- citations ... -->).

Returns:

A vector of trimmed line text strings for unmarked claims.

Signature:

function findUnmarkedClaims(markdown: string): Array<string>

Example:

const result = findUnmarkedClaims("value");

Parameters:

Name	Type	Required	Description
markdown	string	Yes	The markdown text to search

Returns: Array<string>

---

verifyExcerpt()

Verify that an excerpt appears verbatim in source text.

Performs exact matching by default. Also tries whitespace-normalized matching (collapsing runs of whitespace on both sides) since PDF-extracted text often has irregular spacing.

Returns:

true if the excerpt appears (exactly or with normalized whitespace), false otherwise.

Signature:

function verifyExcerpt(excerpt: string, sourceText: string): boolean

Example:

const result = verifyExcerpt("value", "value");

Parameters:

Name	Type	Required	Description
excerpt	string	Yes	The text snippet to find
sourceText	string	Yes	The full source text to search

Returns: boolean

---

maxSimScore()

Score a query against a document using ColBERT’s MaxSim operator: for each query token vector, take the maximum dot product against any document token vector, then sum across query tokens.

Returns 0.0 if query and doc have mismatched dimensionality, if either has zero tokens, or if either is not well-formed per MultiVectorEmbedding.is_well_formed (its data length does not match num_tokens * dim).

Pure CPU primitive — available without ONNX Runtime.

Since v5.0.

Signature:

function maxSimScore(query: MultiVectorEmbedding, doc: MultiVectorEmbedding): number

Example:

const result = maxSimScore(new MultiVectorEmbedding(), new MultiVectorEmbedding());

Parameters:

Name	Type	Required	Description
query	MultiVectorEmbedding	Yes	The multi vector embedding
doc	MultiVectorEmbedding	Yes	The multi vector embedding

Returns: number

---

maxSimRank()

Rank a set of documents against a query by MaxSim score, descending.

Mirrors the sort/truncate shape of crate.reranking’s build_results, minus top-k truncation (callers slice the returned Vec themselves).

Pure CPU primitive — available without ONNX Runtime.

Since v5.0.

Signature:

function maxSimRank(query: MultiVectorEmbedding, docs: Array<MultiVectorEmbedding>): Array<LateInteractionMatch>

Example:

const result = maxSimRank(new MultiVectorEmbedding(), []);

Parameters:

Name	Type	Required	Description
query	MultiVectorEmbedding	Yes	The multi vector embedding
docs	Array<MultiVectorEmbedding>	Yes	The docs

Returns: Array<LateInteractionMatch>

---

doctor()

Probe the backends and settings in config and report what will actually execute on this host.

Runs no downloads and no billable API calls. Backends that are not compiled in or whose models are not cached report Skip rather than failing.

Signature:

function doctor(config: ExtractionConfig): DoctorReport

Example:

const result = doctor(new ExtractionConfig());

Parameters:

Name	Type	Required	Description
config	ExtractionConfig	Yes	The configuration options

Returns: DoctorReport

---

installPdfRenderDiagnostics()

Install PdfOxideWarningCapture as the process-wide log backend, exactly once.

If another component already installed a log.Log implementation (an application wiring env_logger, for instance), log.set_boxed_logger fails and this is a no-op: we do not fight over ownership of the global logger slot, and we do not touch log.set_max_level unless our install won, so we never silently raise or lower a level someone else configured. In that case pdf_oxide’s glyph-drop records go wherever that other logger sends them instead of into take_pdf_oxide_render_warnings. Opt-in. Nothing calls this automatically, and that is deliberate: xberg is a library, and log has exactly one global backend slot per process. A library that claims it on its own behalf breaks its embedder — a host that later calls env_logger.init() panics, and until this returns, every log record in the process is routed here rather than wherever the host intended. That decision belongs to the application, so it is exposed as a call an application makes knowingly.

Returns true if this call (or an earlier one) installed the capture, and false if some other component already owns the log backend — in which case pdf_oxide’s glyph-drop records go to that logger and take_pdf_oxide_render_warnings stays empty.

Without this call the #1364 warnings are not produced. The glyph drop itself is decided inside pdf_oxide, which reports it only through log.warn!; there is no return-value channel to read instead.

Signature:

function installPdfRenderDiagnostics(): boolean

Example:

const result = installPdfRenderDiagnostics();

Returns: boolean

---

takePdfOxideRenderWarnings()

Drain the glyph-drop ProcessingWarnings accumulated on this thread by render calls since the last call to this function.

Callers that render pages as part of extraction should call this after their render pass and merge the result into InternalDocument.processing_warnings (see the module-level convention in crate.core.diagnostics) so a page with missing glyphs is never returned to the user without a signal. Warnings are already deduped per-thread across all pages rendered before this call.

pub (rather than pub(crate)) so both in-tree render-consumers and the regression test for #1364 can observe capture without depending on any one extractor’s internal state.

As of #340, crate.extractors.pdf.mod drains this unconditionally right after assembling a document’s processing_warnings, so every PDF extraction that renders at least one page picks up any captured glyph-drop warnings for free. ~keep: that drain only ever observes warnings from render calls that happened on the same OS thread before it ran, because PDF_OXIDE_PENDING_WARNINGS is thread-local. OCR page rendering runs inline on the extracting task’s thread, so it is covered. Layout-detection rasterization runs inside tokio.task.spawn_blocking, which always executes on a different OS thread, so this function alone would never see those warnings. As of #353, extractors.pdf.layout_runner.run_layout_for_pdf_pages_async drains this function itself from inside its spawn_blocking closure — the only place that can observe the blocking-pool thread’s thread-local buffer — and threads the drained warnings back through its return value for the caller in extractors.pdf.mod to merge, so layout-path glyph drops are no longer silently lost.

Signature:

function takePdfOxideRenderWarnings(): Array<ProcessingWarning>

Example:

const result = takePdfOxideRenderWarnings();

Returns: Array<ProcessingWarning>

---

buildDecoderPromptTokens()

Build the four (or three) token Whisper decoder prompt.

The canonical Whisper prompt is [<|startoftranscript|>, <|{lang}|>, <|transcribe|>, <|notimestamps|>]. When timestamps is true, the trailing no_timestamps token is omitted so the model is free to emit <|x.xx|> timestamp tokens in its output instead of being forced to suppress them.

Signature:

function buildDecoderPromptTokens(startOfTranscript: number, langId: number, transcribe: number, noTimestamps: number, timestamps: boolean): Array<number>

Example:

const result = buildDecoderPromptTokens(42, 42, 42, 42, true);

Parameters:

Name	Type	Required	Description
startOfTranscript	number	Yes	The start of transcript
langId	number	Yes	The lang id
transcribe	number	Yes	The transcribe
noTimestamps	number	Yes	The no timestamps
timestamps	boolean	Yes	The timestamps

Returns: Array<number>

---

timestampTokenToMs()

Convert a raw Whisper timestamp token ID to a millisecond offset from the start of the 30-second chunk it was decoded in.

token_id must be >= timestamp_begin_id; IDs below that are ordinary vocabulary tokens, not timestamps.

Signature:

function timestampTokenToMs(tokenId: number, timestampBeginId: number): number

Example:

const result = timestampTokenToMs(42, 42);

Parameters:

Name	Type	Required	Description
tokenId	number	Yes	The token id
timestampBeginId	number	Yes	The timestamp begin id

Returns: number

---

maxSimRank()

Rank a set of documents against a query by MaxSim score, descending.

Mirrors the sort/truncate shape of crate.reranking’s build_results, minus top-k truncation (callers slice the returned Vec themselves).

Pure CPU primitive — available without ONNX Runtime.

Since v5.0.

Signature:

function maxSimRank(query: MultiVectorEmbedding, docs: Array<MultiVectorEmbedding>): Array<LateInteractionMatch>

Example:

const result = maxSimRank(new MultiVectorEmbedding(), []);

Parameters:

Name	Type	Required	Description
query	MultiVectorEmbedding	Yes	The multi vector embedding
docs	Array<MultiVectorEmbedding>	Yes	The docs

Returns: Array<LateInteractionMatch>

---

maxSimScore()

Score a query against a document using ColBERT’s MaxSim operator: for each query token vector, take the maximum dot product against any document token vector, then sum across query tokens.

Returns 0.0 if query and doc have mismatched dimensionality, if either has zero tokens, or if either is not well-formed per MultiVectorEmbedding.is_well_formed (its data length does not match num_tokens * dim).

Pure CPU primitive — available without ONNX Runtime.

Since v5.0.

Signature:

function maxSimScore(query: MultiVectorEmbedding, doc: MultiVectorEmbedding): number

Example:

const result = maxSimScore(new MultiVectorEmbedding(), new MultiVectorEmbedding());

Parameters:

Name	Type	Required	Description
query	MultiVectorEmbedding	Yes	The multi vector embedding
doc	MultiVectorEmbedding	Yes	The multi vector embedding

Returns: number

---

Types

AccelerationConfig

Hardware acceleration configuration for ONNX Runtime models.

Controls which execution provider (CPU, CoreML, CUDA, TensorRT) is used for inference in layout detection and embedding generation.

Field	Type	Default	Description
provider	ExecutionProviderType	ExecutionProviderType.Auto	Execution provider to use for ONNX inference.
deviceId	number	—	GPU device ID (for CUDA/TensorRT). Ignored for CPU/CoreML/Auto.

---

ArchiveEntry

A single file extracted from an archive.

When archives (ZIP, TAR, 7Z, GZIP) are extracted with recursive extraction enabled, each processable file produces its own full ExtractedDocument.

Field	Type	Default	Description
path	string	—	Archive-relative file path (e.g. “folder/document.pdf”).
mimeType	string	—	Detected MIME type of the file.
result	ExtractedDocument	—	Full extraction result for this file.

---

ArchiveMetadata

Archive (ZIP/TAR/7Z) metadata.

Extracted from compressed archive files containing file lists and size information.

Field	Type	Default	Description
format	string	—	Archive format (“ZIP”, “TAR”, “7Z”, etc.)
fileCount	number	—	Total number of files in the archive
fileList	Array<string>	\[\]	List of file paths within the archive
totalSize	number	—	Total uncompressed size in bytes
compressedSize	number | null	null	Compressed size in bytes (if available)

---

AudioMetadata

Audio/video file metadata.

Populated from container tags (ID3v2, MP4 atoms, Vorbis comments, etc.) and PCM decode properties. Available when the transcription-types feature is enabled.

Field	Type	Default	Description
durationMs	number | null	null	Duration in milliseconds derived from the decoded audio stream.
codec	string | null	null	Audio codec (e.g. “mp3”, “aac”, “opus”, “flac”).
container	string | null	null	Container format (e.g. “mpeg”, “mp4”, “ogg”, “wav”).
sampleRateHz	number | null	null	Sample rate in Hz after decode (always 16000 when resampled for Whisper).
channels	number | null	null	Number of audio channels (1 = mono, 2 = stereo).
bitrate	number | null	null	Audio bitrate in kbps from the source file tags/properties.

---

BBox

Bounding box in original image coordinates (x1, y1) top-left, (x2, y2) bottom-right.

Field	Type	Default	Description
x1	number	—	Left edge (x-coordinate of the top-left corner).
y1	number	—	Top edge (y-coordinate of the top-left corner).
x2	number	—	Right edge (x-coordinate of the bottom-right corner).
y2	number	—	Bottom edge (y-coordinate of the bottom-right corner).

---

BedrockConfig

Since: v1.1

AWS Bedrock configuration for bedrock/-prefixed models.

Mirrors liter-llm’s BedrockConfig. Every field is optional: anything left unset falls back to the standard AWS environment variables (AWS_DEFAULT_REGION / AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, BEDROCK_CROSS_REGION) and the default AWS credential chain. Leave the credential fields unset unless you have an explicit reason to pin them.

Debug is implemented by hand so the three credential fields are never printed.

Field	Type	Default	Description
region	string | null	null	AWS region (e.g. "us-east-1").
crossRegionPrefix	string | null	null	Cross-region inference profile prefix (e.g. "us").
accessKeyId	string | null	null	Explicit AWS access key ID. Secret — never logged.
secretAccessKey	string | null	null	Explicit AWS secret access key. Secret — never logged.
sessionToken	string | null	null	Explicit AWS session token for temporary credentials. Secret — never logged.

---

BibtexMetadata

BibTeX bibliography metadata.

Field	Type	Default	Description
entryCount	number	—	Number of entries in the bibliography.
citationKeys	Array<string>	\[\]	BibTeX citation keys (e.g. "knuth1984") for all entries.
authors	Array<string>	\[\]	Author names collected across all bibliography entries.
yearRange	YearRange | null	null	Earliest and latest publication years found in the bibliography.
entryTypes	Record<string, number> | null	{}	Count of entries grouped by BibTeX entry type (e.g. "article" → 5).

---

BoundingBox

Bounding box coordinates for element positioning.

Field	Type	Default	Description
x0	number	—	Left x-coordinate
y0	number	—	Bottom y-coordinate
x1	number	—	Right x-coordinate
y1	number	—	Top y-coordinate

---

BrowserConfig

Browser fallback configuration.

Field	Type	Default	Description
mode	BrowserMode	BrowserMode.Auto	When to use the headless browser fallback.
backend	BrowserBackend	BrowserBackend.Chromiumoxide	Browser backend used to render JavaScript-heavy pages.
endpoint	string | null	null	CDP WebSocket endpoint for connecting to an external browser instance.
timeout	number	30000ms	Timeout for browser page load and rendering (in milliseconds when serialized).
wait	BrowserWait	BrowserWait.NetworkIdle	Wait strategy after browser navigation.
waitSelector	string | null	null	CSS selector to wait for when wait is Selector.
extraWait	number | null	null	Extra time to wait after the wait condition is met.
proxy	ProxyConfig | null	null	Proxy for browser fetches. Overrides CrawlConfig.proxy when set. Native backend supports http/https only (no SOCKS5).
blockUrlPatterns	Array<string>	\[\]	URL patterns to block before the network request fires. Supports * wildcards. Useful for skipping ads/analytics/large images. Honored by BrowserBackend.Native; chromiumoxide ignores this field today.
evalScript	string | null	null	JavaScript snippet evaluated after navigation completes. Scraping captures the native backend result in ScrapeResult.browser.eval_result. Interactions run this script before page actions on both browser backends but do not include the script result in InteractionResult.
robotsUserAgent	string | null	null	User-agent used when fetching robots.txt. Defaults to BrowserConfig.user_agent (or crawlberg’s default) if unset. Native only.
captureNetworkEvents	boolean	false	Capture the full network event stream into the result. Default false (only the document event is captured). Native only.
sessionAffinity	boolean	true	Enable session affinity: reuse chromiumoxide Pages for same-domain requests so cookies + fingerprint + solved challenges persist. Default: true. When false, each request gets a fresh Page.

---

CacheStats

Aggregate statistics for a xberg cache directory.

Field	Type	Default	Description
totalFiles	number	—	Total number of files currently in the cache directory.
totalSizeMb	number	—	Combined size of all cache files in megabytes.
availableSpaceMb	number	—	Free disk space available on the cache volume, in megabytes.
oldestFileAgeDays	number	—	Age of the oldest cache file in days (0.0 if the cache is empty).
newestFileAgeDays	number	—	Age of the most recently written cache file in days (0.0 if the cache is empty).

---

CaptioningConfig

Since: v1.0

Configuration for the VLM captioning post-processor.

Field	Type	Default	Description
llm	LlmConfig	—	LLM configuration used for the VLM call.
prompt	string | null	null	Optional custom caption prompt. null uses the default RegionKind.Caption prompt that ships with crate.llm.region_extractor.
minImageArea	number	serde(default = "default_min_image_area")	Skip images whose width * height is below this threshold (in pixels). Default 1_000 filters out icons and decorations.

---

CellChange

A single changed cell within a table.

Defined here (rather than only in crate.diff) so RevisionDelta can reference it unconditionally, without requiring the diff Cargo feature. crate.diff re-exports this type verbatim.

Field	Type	Default	Description
row	number	—	Zero-based row index.
col	number	—	Zero-based column index.
from	string	—	Value before the change.
to	string	—	Value after the change.

---

Chunk

A text chunk with optional embedding and metadata.

Chunks are created when chunking is enabled in ExtractionConfig. Each chunk contains the text content, optional embedding vector (if embedding generation is configured), and metadata about its position in the document.

Field	Type	Default	Description
content	string	—	The text content of this chunk.
chunkType	ChunkType	/* serde(default) */	Semantic structural classification of this chunk. Assigned by the heuristic classifier based on content patterns and heading context. Defaults to ChunkType.Unknown when no rule matches.
embedding	Array<number> | null	null	Optional embedding vector for this chunk. Only populated when EmbeddingConfig is provided in chunking configuration. The dimensionality depends on the chosen embedding model.
sparseEmbedding	SparseEmbedding | null	/* serde(default) */	Optional sparse (SPLADE) learned embedding for this chunk. Only populated when sparse-embedding generation is configured for chunking. null otherwise, including on builds without the sparse-embeddings feature. Uses the crate-root SparseEmbedding alias rather than crate.sparse_embeddings.SparseEmbedding directly: the sparse_embeddings module itself only compiles under sparse-embeddings/sparse-embedding-presets, while the crate-root alias is always defined (a field-compatible stub on builds without either feature), so this field — and Chunk itself — compiles on every feature combination, including the crate’s default features.
lateInteraction	MultiVectorEmbedding | null	/* serde(default) */	Optional ColBERT-style multi-vector (late-interaction) embedding for this chunk. Only populated when late-interaction embedding generation is configured for chunking. null otherwise, including on builds without the late-interaction feature. Uses the crate-root MultiVectorEmbedding alias for the same reason sparse_embedding uses SparseEmbedding — see that field’s docs.
metadata	ChunkMetadata	—	Metadata about this chunk’s position and properties.

---

ChunkClassificationConfig

Since: v1.0

Configuration for the chunk-classification post-processor.

Chunk classification is always multi-label: a chunk may match zero, one, or many of the configured definitions. This is the chunk-level equivalent of PageClassificationConfig, but scoped to individual chunks (ExtractedDocument.chunks) rather than whole pages, and built for large taxonomies where each label needs its own description rather than a bare name.

Field	Type	Default	Description
promptTemplate	string | null	null	Minijinja prompt template. Receives {{ definitions }} (rendered label + description list) and {{ chunks }} (a numbered list of chunk texts in the current batch) variables. null lets the backend pick a sensible default.
definitions	Array<ChunkClassificationDefinition>	—	The set of label definitions the classifier may emit. Must contain at least one entry.
llm	LlmConfig	—	LLM configuration used for classification.
batchSize	number	serde(default = "default_batch_size")	Number of chunks batched into a single LLM request. Larger batches amortize the fixed prompt cost (definitions block) across more chunks, at the risk of exceeding the model’s context window for very large taxonomies or chunk texts. Defaults to DEFAULT_BATCH_SIZE.
maxConcurrency	number	serde(default = "default_max_concurrency")	Maximum number of in-flight batch requests. Bounds concurrency against the configured LLM provider. Defaults to DEFAULT_MAX_CONCURRENCY.

---

ChunkClassificationDefinition

Since: v1.0

A single labeled definition the chunk classifier may emit.

Unlike PageClassificationConfig.labels (bare label names), chunk classification targets potentially large domain taxonomies where every label carries its own semantic description, letting the LLM disambiguate similarly named labels without relying on the label string alone.

Field	Type	Default	Description
label	string	—	Label name returned in ChunkMetadata.classifications.
description	string	—	Semantic description of when this label applies. Injected verbatim into the classification prompt next to the label name.

---

ChunkClassificationEnrichmentConfig

Chunk-classification enrichment knob: how to multi-label individual chunks.

Operates on ExtractedDocument.chunks in place — the caller must have already produced chunks (e.g. via ExtractionConfig.chunking) for this stage to have any effect; a document with no chunks is a no-op.

Field	Type	Default	Description
config	ChunkClassificationConfig	—	Label-definition set and LLM/batching settings for the chunk-classification stage.

---

ChunkInfo

Information about a single chunk.

Field	Type	Default	Description
index	number	—	Zero-based chunk index.
pages	PageRange	—	Page range for this chunk.
estimatedTimeMs	number	—	Estimated processing time for this chunk in milliseconds.

---

ChunkMetadata

Metadata about a chunk’s position in the original document.

Field	Type	Default	Description
byteStart	number	—	Byte offset where this chunk starts in the original text (UTF-8 valid boundary).
byteEnd	number	—	Byte offset where this chunk ends in the original text (UTF-8 valid boundary).
tokenCount	number | null	null	Number of tokens in this chunk (if available). This is calculated by the embedding model’s tokenizer if embeddings are enabled.
chunkIndex	number	—	Zero-based index of this chunk in the document.
totalChunks	number	—	Total number of chunks in the document.
firstPage	number | null	null	First page number this chunk spans (1-indexed). Only populated when page tracking is enabled in extraction configuration.
lastPage	number | null	null	Last page number this chunk spans (1-indexed, equal to first_page for single-page chunks). Only populated when page tracking is enabled in extraction configuration.
headingContext	HeadingContext | null	/* serde(default) */	Heading context when using Markdown chunker. Contains the heading hierarchy this chunk falls under. Only populated when ChunkerType.Markdown is used.
headingPath	Array<string>	/* serde(default) */	Flattened heading trail from document root to this chunk’s section. Each element is a heading’s text, outermost first. Derived from heading_context when present; empty otherwise. Provides a binding-friendly, RAG-shaped breadcrumb without requiring callers to walk the nested HeadingContext structure.
imageIndices	Array<number>	/* serde(default) */	Indices into ExtractedDocument.images for images on pages covered by this chunk. Contains zero-based indices into the top-level images collection for every image whose page_number falls within \[first_page, last_page\]. Empty when image extraction is disabled or the chunk spans no pages with images.
nodeIds	Array<string>	/* serde(default) */	Ids of the DocumentNodes this chunk was derived from. Joins a chunk back to the structured document tree via DocumentNode.id. Empty until the node-to-rendered-offset mapping needed to compute the intersection is implemented (tracked under #1294/#1295); this field is the wire-format foundation for that follow-up.
pageSpans	Array<PageSpan>	/* serde(default) */	Per-page bounding-box spans this chunk covers, for viewer highlighting (#1295). One entry per page the chunk overlaps, in page order — the first and last entries’ page fields equal first_page/last_page. Populated whenever page-boundary provenance is available (the same condition under which first_page/last_page are populated); each entry’s bbox is additionally populated when the document’s structured node tree (ExtractedDocument.document) is available, as the union of that page’s body-layer node bounding boxes found within this chunk. Empty when page-boundary provenance is unavailable (mirrors first_page/ last_page being null).
classifications	Array<ClassificationLabel>	/* serde(default) */	Multi-label classification result for this chunk. Populated by the chunk-classification post-processor when ExtractionConfig.chunk_classification is set. A chunk may match zero, one, or many of the configured label definitions. Empty when chunk classification was not configured.

---

ChunkingConfig

Chunking configuration.

Configures text chunking for document content, including chunk size, overlap, trimming behavior, and optional embeddings.

Use ..the default constructor when constructing to allow for future field additions:

Field	Type	Default	Description
maxCharacters	number	1000	Maximum size per chunk (in units determined by sizing). When sizing is Characters (default), this is the max character count. When using token-based sizing, this is the max token count. Default: 1000
overlap	number	200	Overlap between chunks (in units determined by sizing). Default: 200
trim	boolean	true	Whether to trim whitespace from chunk boundaries. Default: true
chunkerType	ChunkerType	ChunkerType.Text	Type of chunker to use (Text or Markdown). Default: Text
embedding	EmbeddingConfig | null	null	Optional embedding configuration for chunk embeddings.
sparseEmbedding	SparseEmbeddingConfig | null	null	Optional sparse (SPLADE) embedding configuration for chunk embeddings. When set, sparse vectors are generated for each chunk’s content and attached via sparse_embedding. Requires the sparse-embeddings feature; without it, a warning is emitted and no sparse vectors are attached. Config-file only: like RerankerConfig and the local-ONNX branch of embedding, this has no CLI flag and no environment variable. Only the secret/identity fields of LLM-routed configs (model, API key, base URL) get that reach. ~keep
lateInteraction	LateInteractionConfig | null	null	Optional late-interaction (ColBERT) embedding configuration for chunk embeddings. When set, multi-vector embeddings are generated for each chunk’s content and attached via late_interaction. Requires the late-interaction feature; without it, a warning is emitted and no late-interaction vectors are attached. Config-file only, for the same reason as sparse_embedding above. ~keep
preset	string | null	null	Use a preset configuration (overrides individual settings if provided).
sizing	ChunkSizing	ChunkSizing.Characters	How to measure chunk size. Default: Characters (Unicode character count). Enable chunking-tiktoken or chunking-tokenizers features for token-based sizing.
prependHeadingContext	boolean	false	Deprecated and inert (#1393): no longer prepends anything into content. Setting this field has no observable effect on chunking output any more. Previously, when true and chunker_type was Markdown, this prepended the heading hierarchy path (e.g. "# Title > ## Section\n\n") directly into each chunk’s content string. content now always equals the exact \[byte_start, byte_end) source span regardless of this flag — see BreadcrumbTarget for the full rationale. heading_context/heading_path on ChunkMetadata are populated independently of this flag, so callers lose no information — only the in-place mutation is gone. Call render_heading_breadcrumb explicitly at index time instead, for the retrieval consumer that wants the breadcrumb inline. Kept only so existing callers keep compiling. Default: false
topicThreshold	number | null	null	Optional cosine similarity threshold for semantic topic boundary detection. Only used when chunker_type is Semantic and an EmbeddingConfig is provided. You almost never need to set this. When omitted, defaults to 0.75 which works well for most documents. Lower values detect more topic boundaries (more, smaller chunks); higher values detect fewer. Range: 0.0..=1.0.
tableChunking	TableChunkingMode	TableChunkingMode.Split	How to handle markdown tables that exceed the chunk size limit. Only applies when chunker_type is Markdown. - Split (default) — tables are split at row boundaries; continuation chunks do not repeat the header. - RepeatHeader — the table header row and separator are prepended to every continuation chunk so each chunk is self-contained. Default: Split
breadcrumbTarget	BreadcrumbTarget	BreadcrumbTarget.Content	Deprecated and inert (#1393): see BreadcrumbTarget for the full explanation. Neither variant has any effect on content any more — call render_heading_breadcrumb explicitly at index time instead. Kept only for backward compatibility. Default: Content.

Methods

default()

Signature:

static default(): ChunkingConfig

Example:

const result = ChunkingConfig.default();

Returns: ChunkingConfig

---

Citation

A structured citation from a citation block.

Parsed from entries like: [^srcN]: source, locator, excerpt: "text"

Field	Type	Default	Description
label	string	—	The label of the citation (e.g., “src1” in \[^src1\]: ...).
source	string	—	The source reference (path, URL, or identifier).
locator	string | null	null	Optional locator within the source (e.g., “page 3” or “section 2.1”).
excerpt	string | null	null	Optional excerpt — quoted text from the source.

---

CitationMetadata

Citation file metadata (RIS, PubMed, EndNote).

Field	Type	Default	Description
citationCount	number	—	Total number of citation records in the file.
format	string | null	null	Detected citation file format (e.g. "ris", "pubmed", "endnote").
authors	Array<string>	\[\]	Author names collected across all citation records.
yearRange	YearRange | null	null	Earliest and latest publication years found in the file.
dois	Array<string>	\[\]	DOI identifiers found in the citation records.
keywords	Array<string>	\[\]	Keywords collected from all citation records.

---

ClassificationLabel

A single label + confidence pair.

Field	Type	Default	Description
label	string	—	Label name as configured in PageClassificationConfig.labels.
confidence	number | null	null	Backend-reported confidence in \[0.0, 1.0\]. null when the backend (e.g. an LLM prompt without explicit confidence schema) did not report one.

---

CodeChunkInfo

A single structurally-meaningful code chunk produced by tree-sitter parsing.

Purpose-built payload owned by xberg — deliberately does not expose the upstream tree_sitter_language_pack types, so binding generators never need to resolve an external crate’s types across FFI/language boundaries.

Field	Type	Default	Description
text	string	—	The raw source text of this chunk.
contextPath	Array<string>	—	Hierarchical path of enclosing structural items (e.g. \["MyClass", "my_method"\]).
nodeTypes	Array<string>	—	Tree-sitter node kinds that appear at the top level of this chunk (e.g. "function_definition", "class_definition").
byteStart	number	—	Inclusive start byte offset of this chunk in the original source.
byteEnd	number	—	Exclusive end byte offset of this chunk in the original source.

---

CodeDataAttribute

An XML-style attribute attached to an Element node.

Populated only for CodeDataNodeKind.Element; always empty for KeyValue and Sequence nodes.

Field	Type	Default	Description
name	string	—	Attribute name (e.g. "class", "href").
value	string	—	Attribute value as a raw string (quotes stripped).
byteStart	number	—	Inclusive start byte offset of the name="value" attribute token.
byteEnd	number	—	Exclusive end byte offset of the name="value" attribute token.

---

CodeDataNode

A node in the hierarchical data tree produced by data-format extraction.

Purpose-built payload owned by xberg — mirrors tree_sitter_language_pack.DataNode but flattens its Span down to plain byte offsets, so binding generators never need to resolve an external crate’s types across FFI/language boundaries.

Field	Type	Default	Description
kind	CodeDataNodeKind	—	Whether this node is a key/value pair, XML element, or sequence item.
key	string | null	/* serde(default) */	Key, attribute name, tag name, or positional index ("0", "1", …). null at the document root.
value	string | null	/* serde(default) */	Leaf scalar value, if any. null for containers (objects, arrays, XML elements with child elements).
attributes	Array<CodeDataAttribute>	/* serde(default) */	Attributes on element-shape nodes (XML STag attributes). Empty for all other kinds.
children	Array<CodeDataNode>	/* serde(default) */	Children for nested containers and XML element bodies.
byteStart	number	—	Inclusive start byte offset of this node in the original source.
byteEnd	number	—	Exclusive end byte offset of this node in the original source.

---

CodeMetadata

Code-format metadata: the structural chunks produced by tree-sitter parsing.

Wrapped by FormatMetadata.Code. Kept as a named struct (rather than an inline enum-variant body) so serde can tag it under internal tagging and utoipa can emit a referenceable CodeMetadata component in the OpenAPI schema.

Field	Type	Default	Description
chunks	Array<CodeChunkInfo>	\[\]	Structural code chunks (function/class/module boundaries).
data	CodeDataNode | null	null	Hierarchical key/value data tree extracted from data-format source (JSON, YAML, TOML, XML, CSV, etc.), when data extraction was enabled.

---

ContentConfig

Content extraction and conversion configuration.

Controls how HTML is converted to the output format. Uses html-to-markdown-rs as the conversion engine for all formats (markdown, plain text, djot).

Field	Type	Default	Description
outputFormat	string	"markdown"	Output format: "markdown" (default), "plain", "djot".
preprocessingPreset	string	"standard"	Preprocessing aggressiveness: "minimal", "standard" (default), "aggressive". - Minimal: only scripts/styles removed. - Standard: also removes nav, nav-hinted headers/footers/asides, forms. - Aggressive: removes all footers/asides unconditionally.
removeNavigation	boolean	true	Remove navigation elements (nav, breadcrumbs, menus). Default: true.
removeForms	boolean	true	Remove form elements. Default: true.
stripTags	Array<string>	\[\]	HTML tag names to strip (render children only, remove the tag wrapper). Default: \["noscript"\].
preserveTags	Array<string>	\[\]	HTML tag names to preserve as raw HTML in output.
excludeSelectors	Array<string>	\[\]	CSS selectors for elements to exclude entirely (element + all content). Unlike strip_tags (which removes the wrapper but keeps children), excluded elements and all descendants are dropped. Supports CSS selectors: .class, #id, \[attribute\], compound selectors. Example: \[".cookie-banner", "#ad-container", "\[role='complementary'\]"\]
skipImages	boolean	false	Skip image elements in output. Default: false.
maxDepth	number | null	null	Max DOM traversal depth. Prevents stack overflow on deeply nested HTML.
wrap	boolean	false	Enable line wrapping. Default: false.
wrapWidth	number	80	Wrap width when wrap is enabled. Default: 80.
includeDocumentStructure	boolean	true	Include document structure tree in output. Default: true.

---

ContentFilterConfig

Cross-extractor content filtering configuration.

Controls whether “furniture” content (headers, footers, page numbers, watermarks, repeating text) is included in or stripped from extraction results. Applies across all extractors (PDF, DOCX, RTF, ODT, HTML, etc.) with format-specific implementation.

When null on ExtractionConfig, each extractor uses its current default behavior unchanged.

Field	Type	Default	Description
includeHeaders	boolean	false	Include running headers in extraction output. - PDF: Disables top-margin furniture stripping and prevents the layout model from treating PageHeader-classified regions as furniture. - DOCX: Includes document headers in text output. - RTF/ODT: Headers already included; this is a no-op when true. - HTML/EPUB: Keeps <header> element content. Default: false (headers are stripped or excluded).
includeFooters	boolean	false	Include running footers in extraction output. - PDF: Disables bottom-margin furniture stripping and prevents the layout model from treating PageFooter-classified regions as furniture. - DOCX: Includes document footers in text output. - RTF/ODT: Footers already included; this is a no-op when true. - HTML/EPUB: Keeps <footer> element content. Default: false (footers are stripped or excluded).
includeFootnotes	boolean	false	Include footnote bodies in extraction output. - PDF: Prevents the layout model from treating Footnote-classified regions as furniture, so footnote bodies survive alongside the main text instead of being silently dropped. - Other formats: No effect currently. Default: false (footnotes are stripped), matching the existing include_headers / include_footers defaults.
stripRepeatingText	boolean	true	Enable the heuristic cross-page repeating text detector. When true (default), text that repeats verbatim across a supermajority of pages is classified as furniture and stripped. Disable this if brand names or repeated headings are being incorrectly removed by the heuristic. Note: when a layout-detection model is active, the model may independently classify page-header / page-footer / footnote regions as furniture on a per-page basis. To preserve those regions, set include_headers = true, include_footers = true, include_footnotes = true, or any combination, in addition to disabling this flag. Primarily affects PDF extraction. Default: true.
includeWatermarks	boolean	false	Include watermark text in extraction output. - PDF: Keeps watermark artifacts and arXiv identifiers. - Other formats: No effect currently. Default: false (watermarks are stripped).

Methods

default()

Signature:

static default(): ContentFilterConfig

Example:

const result = ContentFilterConfig.default();

Returns: ContentFilterConfig

---

ContributorRole

JATS contributor with role.

Field	Type	Default	Description
name	string	—	Contributor display name.
role	string | null	null	Contributor role (e.g. "author", "editor").

---

ConversionOptions

Main conversion options for HTML to Markdown conversion.

Use ConversionOptions.builder() to construct, or the default constructor for defaults.

Field	Type	Default	Description
headingStyle	HeadingStyle	HeadingStyle.Atx	Heading style to use in Markdown output (ATX # or Setext underline).
listIndentType	ListIndentType	ListIndentType.Spaces	How to indent nested list items (spaces or tab).
listIndentWidth	number	2	Number of spaces (or tabs) to use for each level of list indentation.
bullets	string	"-*+"	Bullet character(s) to use for unordered list items (e.g. "-", "*").
strongEmSymbol	string	"*"	Character used for bold/italic emphasis markers (* or _).
escapeAsterisks	boolean	false	Escape * characters in plain text to avoid unintended bold/italic.
escapeUnderscores	boolean	false	Escape _ characters in plain text to avoid unintended bold/italic.
escapeMisc	boolean	false	Escape miscellaneous Markdown metacharacters (\[\]()# etc.) in plain text.
escapeAscii	boolean	false	Escape ASCII characters that have special meaning in certain Markdown dialects.
codeLanguage	string	""	Default language annotation for fenced code blocks that have no language hint.
autolinks	boolean	true	Automatically convert bare URLs into Markdown autolinks.
defaultTitle	boolean	false	Emit a default title when no <title> tag is present.
brInTables	boolean	false	Render <br> elements inside table cells as literal line breaks.
compactTables	boolean	false	Emit tables without column padding (compact GFM format). When true, column widths are not computed and cells are emitted with no trailing spaces. Separator rows use exactly --- per column. Produces token-efficient output suitable for RAG / LLM contexts. Default false (aligned padding preserved).
highlightStyle	HighlightStyle	HighlightStyle.DoubleEqual	Style used for <mark> / highlighted text (e.g. ==text==).
extractMetadata	boolean	true	Populate result.metadata with <head> / <meta> extraction (title, description, Open Graph, Twitter Card, JSON-LD, …). Default true. Disabling skips the metadata pass only — table extraction into result.tables runs unconditionally.
whitespaceMode	WhitespaceMode	WhitespaceMode.Normalized	Controls how whitespace sequences are normalised in the converted output. - WhitespaceMode.Normalized (default) — collapses consecutive whitespace characters (spaces, tabs, newlines) to a single space, matching browser rendering behaviour. - WhitespaceMode.Strict — preserves all whitespace exactly as it appears in the source HTML, including runs of spaces and embedded newlines. Choose Strict only when the source HTML uses deliberate whitespace (e.g. pre-formatted content outside <pre> tags). For most documents Normalized produces cleaner output.
stripNewlines	boolean	false	Strip all newlines from the output, producing a single-line result.
wrap	boolean	false	Wrap long lines at wrap_width characters.
wrapWidth	number	80	Maximum output line width in characters when wrap is true (default 80). Lines are broken at word boundaries so that no line exceeds this length. A value of 0 is treated as “no limit” — equivalent to leaving wrap disabled. Has no effect when wrap is false.
convertAsInline	boolean	false	Treat the entire document as inline content (no block-level wrappers).
subSymbol	string	""	Markdown notation for subscript text (e.g. "~").
supSymbol	string	""	Markdown notation for superscript text (e.g. "^").
newlineStyle	NewlineStyle	NewlineStyle.Spaces	How to encode hard line breaks (<br>) in Markdown.
codeBlockStyle	CodeBlockStyle	CodeBlockStyle.Backticks	Style used for fenced code blocks (backticks or tilde).
keepInlineImagesIn	Array<string>	\[\]	HTML tag names whose <img> children are kept inline instead of block.
preprocessing	PreprocessingOptions	—	Options for the HTML pre-processing pass applied before conversion begins. Pre-processing runs before the HTML is handed to the converter and can perform operations such as unwrapping redundant wrapper elements, removing tracking pixels, and normalising vendor-specific markup. See PreprocessingOptions for the full set of knobs. Defaults to the standard preprocessing options, which enables the standard cleaning passes. Set individual fields on PreprocessingOptions (or construct via ConversionOptions.builder) to opt in or out of specific passes.
encoding	string	"utf-8"	Expected character encoding of the input HTML (default "utf-8").
debug	boolean	false	Emit debug information during conversion.
stripTags	Array<string>	\[\]	HTML tag names whose content is stripped from the output entirely.
preserveTags	Array<string>	\[\]	HTML tag names that are preserved verbatim in the output.
skipImages	boolean	false	Skip conversion of <img> elements (omit images from output).
urlEscapeStyle	UrlEscapeStyle	UrlEscapeStyle.Angle	URL encoding strategy for link and image destinations. Controls how special characters in URL destinations are escaped: - UrlEscapeStyle.Angle (default) — wraps the destination in angle brackets when it contains spaces or newlines. Some parsers misinterpret > inside such a destination. - UrlEscapeStyle.Percent — percent-encodes every character that is not an RFC 3986 unreserved character or /, producing a destination that all Markdown parsers handle correctly even when the URL contains <, >, spaces, or parentheses.
linkStyle	LinkStyle	LinkStyle.Inline	Link rendering style (inline or reference).
maxImageSize	number	5242880	Maximum decoded image size in bytes (default 5MB).
captureSvg	boolean	false	Capture SVG elements as images.
inferDimensions	boolean	true	Infer image dimensions from data.
maxDepth	number | null	null	Maximum DOM traversal depth. null uses the library’s internal native-stack safety limit. Explicit values above that safety limit are clamped to prevent process-aborting stack overflows on pathologically deep DOM trees.
excludeSelectors	Array<string>	\[\]	CSS selectors for elements to exclude entirely (element + all content). Unlike strip_tags (which removes the tag wrapper but keeps children), excluded elements and all their descendants are dropped from the output. Supports any CSS selector that tl supports: tag names, .class, #id, \[attribute\], etc. Invalid selectors are silently skipped at conversion time. Example: \[".cookie-banner", "#ad-container", "\[role='complementary'\]"\]

---

CoreProperties

Dublin Core metadata from docProps/core.xml

Contains standard metadata fields defined by the Dublin Core standard and Office-specific extensions.

Field	Type	Default	Description
title	string | null	null	Document title
subject	string | null	null	Document subject/topic
creator	string | null	null	Document creator/author
keywords	string | null	null	Keywords or tags
description	string | null	null	Document description/abstract
lastModifiedBy	string | null	null	User who last modified the document
revision	string | null	null	Revision number
created	string | null	null	Creation timestamp (ISO 8601)
modified	string | null	null	Last modification timestamp (ISO 8601)
category	string | null	null	Document category
contentStatus	string | null	null	Content status (Draft, Final, etc.)
language	string | null	null	Document language
identifier	string | null	null	Unique identifier
version	string | null	null	Document version
lastPrinted	string | null	null	Last print timestamp (ISO 8601)

---

CrawlConfig

Configuration for crawl, scrape, and map operations.

Field	Type	Default	Description
maxDepth	number | null	null	Maximum crawl depth (number of link hops from the start URL).
maxPages	number | null	null	Maximum number of pages to crawl.
maxConcurrent	number | null	null	Maximum number of concurrent requests.
respectRobotsTxt	boolean	false	Whether to respect robots.txt directives.
softHttpErrors	boolean	false	When true, HTTP-level error responses (404 NotFound, 403 Forbidden, WAF blocks) are surfaced as ScrapeResult records with the matching status_code rather than raised as CrawlError. Default false preserves the historical throw-on-error contract for direct fetches. Independently of this flag, 404s reached at the end of a redirect chain are always surfaced softly — the user opted into redirect-following, so receiving a 404 there is part of the normal flow rather than an unexpected error.
userAgent	string | null	null	Custom user-agent string.
stayOnDomain	boolean	false	Whether to restrict crawling to the same domain.
allowSubdomains	boolean	false	Whether to allow subdomains when stay_on_domain is true.
includePaths	Array<string>	\[\]	Regex patterns for paths to include during crawling.
excludePaths	Array<string>	\[\]	Regex patterns for paths to exclude during crawling.
customHeaders	Record<string, string>	{}	Custom HTTP headers to send with each request.
requestTimeout	number	30000ms	Timeout for individual HTTP requests (in milliseconds when serialized).
rateLimitMs	number | null	null	Per-domain rate limit in milliseconds. When set, enforces a minimum delay between requests to the same domain. Defaults to 200ms when null.
maxRedirects	number	10	Maximum number of redirects to follow.
retryCount	number	0	Number of retry attempts for failed requests.
retryCodes	Array<number>	\[\]	HTTP status codes that should trigger a retry.
cookiesEnabled	boolean	false	Whether to enable cookie handling.
auth	AuthConfig | null	null	Authentication configuration.
maxBodySize	number | null	null	Maximum response body size in bytes.
removeTags	Array<string>	\[\]	CSS selectors for tags to remove from HTML before processing.
content	ContentConfig	—	Content extraction and conversion configuration.
mapLimit	number | null	null	Maximum number of URLs to return from a map operation.
mapSearch	string | null	null	Search filter for map results (case-insensitive substring match on URLs).
downloadAssets	boolean	false	Whether to download assets (CSS, JS, images, etc.) from the page.
assetTypes	Array<AssetCategory>	\[\]	Filter for asset categories to download.
maxAssetSize	number | null	null	Maximum size in bytes for individual asset downloads.
browser	BrowserConfig	—	Browser configuration.
proxy	ProxyConfig | null	null	Proxy configuration for HTTP requests.
userAgents	Array<string>	\[\]	List of user-agent strings for rotation. If non-empty, overrides user_agent.
captureScreenshot	boolean	false	Whether to capture a screenshot when using the browser.
followDocumentUrls	boolean	false	Re-enqueue discovered LinkType.Document URLs into the crawl frontier so the crawl follows links from document pages (PDFs, etc.) as it would from HTML pages. Default: false (documents terminate at materialisation).
documentUrlDepth	number | null	null	Maximum document-depth (from the seed URL through document links only) when follow_document_urls is true. null means inherit max_depth. Independent of max_depth: a document URL is enqueued only if BOTH the outer max_depth and (if set) document_url_depth permit it.
downloadDocuments	boolean	true	Whether to download non-HTML documents (PDF, DOCX, images, code, etc.) instead of skipping them.
documentMaxSize	number | null	52428800	Maximum size in bytes for document downloads. Defaults to 50 MB.
documentMimeTypes	Array<string>	\[\]	Allowlist of MIME types to download. If empty, uses built-in defaults.
warcOutput	string | null	null	Path to write WARC output. If null, WARC output is disabled.
browserProfile	string | null	null	Named browser profile for persistent sessions (cookies, localStorage).
saveBrowserProfile	boolean	false	Whether to save changes back to the browser profile on exit.
ssrf	SsrfPolicy	—	SSRF policy for outbound network requests. Default: deny private networks, allow http/https only, max 5 redirects. Phase 1: deny_private and max_redirects are exposed to all language bindings. allowlist is skipped (see SsrfPolicy fields) and will be added in a follow-up when HostMatcher’s tagged-enum FFI form is decided.

---

CsvConfig

Since: v1.1

Configuration for CSV/TSV extraction.

When unset (ExtractionConfig.csv == None), the extractor keeps its existing default behavior: the delimiter is auto-detected by sampling the file (comma, tab, pipe, or semicolon), and no line is treated as a comment.

Field	Type	Default	Description
delimiter	string | null	null	Field delimiter, as a single-character string (e.g. ",", ";", "\t", "|"). When null (default), the delimiter is auto-detected from a sample of the file. Must be exactly one ASCII byte when set — ExtractionConfig.validate rejects an empty string or a multi-byte value with a helpful error. The TSV MIME type (text/tab-separated-values) always forces \t regardless of this setting.
commentPrefixes	Array<string>	\[\]	Line prefixes that mark a comment line to skip entirely during row parsing (e.g. \["#"\]). A line is treated as a comment when its trimmed start matches any of these prefixes exactly. Default: empty, meaning no line is treated as a comment (matches the pre-existing extractor behavior).

---

CsvMetadata

CSV/TSV file metadata.

Field	Type	Default	Description
rowCount	number	—	Total number of data rows (excluding the header row if present).
columnCount	number	—	Number of columns detected.
delimiter	string | null	null	Field delimiter character (e.g. "," or "\t").
hasHeader	boolean	—	Whether the first row was treated as a header.
columnTypes	Array<string> | null	\[\]	Inferred data type for each column (e.g. "string", "integer", "float").

---

DbfFieldInfo

dBASE field information.

Field	Type	Default	Description
name	string	—	Field (column) name.
fieldType	string	—	dBASE field type character (e.g. "C" for character, "N" for numeric).

---

DbfMetadata

dBASE (DBF) file metadata.

Field	Type	Default	Description
recordCount	number	—	Total number of data records in the DBF file.
fieldCount	number	—	Number of field (column) definitions.
fields	Array<DbfFieldInfo>	\[\]	Descriptor for each field in the table schema.

---

DetectResponse

MIME type detection response.

Field	Type	Default	Description
mimeType	string	—	Detected MIME type
filename	string | null	null	Original filename (if provided)

---

DetectionResult

Page-level detection result containing all detections and page metadata.

Field	Type	Default	Description
pageWidth	number	—	Page width in pixels (as seen by the model).
pageHeight	number	—	Page height in pixels (as seen by the model).
detections	Array<LayoutDetection>	—	All layout detections on this page after postprocessing.

---

DiffHunk

A single contiguous hunk in a unified diff.

Field	Type	Default	Description
fromLine	number	—	Starting line number in the old content (0-indexed).
fromCount	number	—	Number of lines from the old content in this hunk.
toLine	number	—	Starting line number in the new content (0-indexed).
toCount	number	—	Number of lines from the new content in this hunk.
lines	Array<DiffLine>	—	Lines that make up this hunk.

---

DiffOptions

Options controlling how two ExtractedDocument values are compared.

Field	Type	Default	Description
includeMetadata	boolean	true	Include metadata changes in the diff. Default: true.
includeEmbedded	boolean	true	Include embedded-children changes in the diff. Default: true.
maxContentChars	number | null	null	Truncate content to this many characters before diffing. Useful for very large documents where only the first N characters matter. null means no truncation.

Methods

default()

Signature:

static default(): DiffOptions

Example:

const result = DiffOptions.default();

Returns: DiffOptions

---

DjotContent

Comprehensive Djot document structure with semantic preservation.

This type captures the full richness of Djot markup, including:

• Block-level structures (headings, lists, blockquotes, code blocks, etc.)
• Inline formatting (emphasis, strong, highlight, subscript, superscript, etc.)
• Attributes (classes, IDs, key-value pairs)
• Links, images, footnotes
• Math expressions (inline and display)
• Tables with full structure

Available when the djot feature is enabled.

Field	Type	Default	Description
plainText	string	—	Plain text representation for backwards compatibility
blocks	Array<FormattedBlock>	—	Structured block-level content
metadata	Metadata	—	Metadata from YAML frontmatter
tables	Array<Table>	—	Extracted tables as structured data
images	Array<DjotImage>	—	Extracted images with metadata
links	Array<DjotLink>	—	Extracted links with URLs
footnotes	Array<Footnote>	—	Footnote definitions

---

DjotImage

Image element in Djot.

Field	Type	Default	Description
src	string	—	Image source URL or path
alt	string	—	Alternative text
title	string | null	null	Optional title

---

DjotLink

Link element in Djot.

Field	Type	Default	Description
url	string	—	Link URL
text	string	—	Link text content
title	string | null	null	Optional title

---

DoctorCheck

A single doctor verdict: what was checked, the outcome, and why.

Field	Type	Default	Description
name	string	—	Check identifier, e.g. ocr.tesseract or layout.rtdetr.
status	ProbeStatus	—	Pass / warn / fail / skip verdict.
message	string	—	One-line reason or detail (e.g. missing language, resolved path, error).

Methods

pass()

A ProbeStatus.Pass verdict: the checked backend or setting will work as configured on this host.

Signature:

static pass(name: string, message: string): DoctorCheck

Example:

const result = DoctorCheck.pass("value", "value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name
message	string	Yes	The message

Returns: DoctorCheck

warn()

A ProbeStatus.Warn verdict: the check ran and found something actionable, but nothing is broken. Never fails the report.

Signature:

static warn(name: string, message: string): DoctorCheck

Example:

const result = DoctorCheck.warn("value", "value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name
message	string	Yes	The message

Returns: DoctorCheck

fail()

A ProbeStatus.Fail verdict: the configured setup will not work (or will silently degrade) on this host.

Signature:

static fail(name: string, message: string): DoctorCheck

Example:

const result = DoctorCheck.fail("value", "value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name
message	string	Yes	The message

Returns: DoctorCheck

skip()

A ProbeStatus.Skip verdict: the check cannot run locally; first real use decides, possibly after a download.

Signature:

static skip(name: string, message: string): DoctorCheck

Example:

const result = DoctorCheck.skip("value", "value");

Parameters:

Name	Type	Required	Description
name	string	Yes	The name
message	string	Yes	The message

Returns: DoctorCheck

---

DoctorReport

Aggregate doctor report over all configured backends and settings.

Field	Type	Default	Description
checks	Array<DoctorCheck>	\[\]	Individual check verdicts, in execution order.

Methods

isOk()

Whether no check failed. Warnings and skips do not count as failures, so the report stays usable as a scripting/CI gate.

Signature:

isOk(): boolean

Example:

const result = instance.isOk();

Returns: boolean

---

DocumentBoundary

Detected document boundary within a PDF.

Field	Type	Default	Description
startPage	number	—	1-indexed start page (inclusive).
endPage	number	—	1-indexed end page (inclusive).
confidence	number	—	Confidence in this boundary, \[0.0, 1.0\].
reason	BoundaryReason	—	Reason for the boundary detection.

---

DocumentCounts

Cheap structural counts for an extracted document.

Populated on every ExtractedDocument returned by extract / extract_batch, regardless of whether the heavy pages / images collections are materialized. A caller that only needs “how many pages / tables / images did this document have?” (reporting, cost estimation, progress, quotas) can read these without enabling per-page or per-image extraction.

The page count comes from the parse (the extractor already walks the page tree); it does not require opting into per-page content. pages is 0 for inputs that are not page-addressable (e.g. plain text).

Field	Type	Default	Description
pages	number	—	Total pages in the source document (0 when not page-addressable).
tables	number	—	Tables detected in the document.
images	number	—	Images detected in the document.

---

DocumentExtractor

Trait for document extractor plugins.

Implement this trait to add support for new document formats or override built-in extraction behavior. Foreign-language bindings expose the DocumentExtractor.extract method, which accepts ExtractInput and returns an ExtractedDocument.

Priority System

When multiple extractors support the same MIME type, the registry selects the extractor with the highest priority value. Use this to:

• Override built-in extractors (priority > 50)
• Provide fallback extractors (priority < 50)
• Implement specialized extractors for specific use cases

Default priority is 50.

Thread Safety

Extractors must be thread-safe (Send + Sync) to support concurrent extraction.

Methods

extract()

Binding-safe extraction entry point for foreign-language plugin bridges.

Accepts the same unified input shape as the public extraction API and returns one extracted document result.

Signature:

extract(input: ExtractInput, config: ExtractionConfig): Promise<ExtractedDocument>

Example:

const result = await instance.extract(new ExtractInput(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
input	ExtractInput	Yes	The input data
config	ExtractionConfig	Yes	The configuration options

Returns: ExtractedDocument

Errors: Throws Error with a descriptive message.

supportedMimeTypes()

Get the list of MIME types supported by this extractor.

Can include exact MIME types and prefix patterns:

• Exact: "application/pdf", "text/plain"
• Prefix: "image/*" (matches any image type)

Returns:

A slice of MIME type strings.

Signature:

supportedMimeTypes(): Array<string>

Example:

const result = instance.supportedMimeTypes();

Returns: Array<string>

priority()

Get the priority of this extractor.

Higher priority extractors are preferred when multiple extractors support the same MIME type.

Priority Guidelines

• 0-25: Fallback/low-quality extractors
• 26-49: Alternative extractors
• 50: Default priority (built-in extractors)
• 51-75: Premium/enhanced extractors
• 76-100: Specialized/high-priority extractors

Returns:

Priority value (default: 50)

Signature:

priority(): number

Example:

const result = instance.priority();

Returns: number

canHandle()

Optional: Check if this extractor can handle a specific file.

Allows for more sophisticated detection beyond MIME types. Defaults to true (rely on MIME type matching).

Returns:

true if the extractor can handle this file, false otherwise.

Signature:

canHandle(path: string, mimeType: string): boolean

Example:

const result = instance.canHandle("value", "value");

Parameters:

Name	Type	Required	Description
path	string	Yes	The path
mimeType	string	Yes	The mime type

Returns: boolean

---

DocumentMetadata

Metadata about a document for analysis.

Field	Type	Default	Description
mimeType	string	—	MIME type of the document.
sizeBytes	number	—	File size in bytes.
pageCount	number | null	null	Page count (if known, e.g., from previous analysis).
forceOcr	boolean	—	Whether OCR is forced regardless of text layer.
userChunkConfig	UserChunkConfig | null	null	User-provided chunk configuration overrides.
chunkingEnabled	boolean	—	Whether chunking is enabled for this job.

---

DocumentNode

A single node in the document tree.

Each node has deterministic id, typed content, optional parent/children for tree structure, and metadata like page number, bounding box, and content layer.

Field	Type	Default	Description
id	string	/* serde(default) */	Deterministic identifier (hash of node type + text + page + position). Stable and unique within a single extraction response: every internal construction path threads the node’s position (its index in DocumentStructure.nodes) into the hash, so identical (node_type, text, page) tuples at different positions never collide. Always serialised — ChunkMetadata.node_ids references it to join chunks back to the nodes they were derived from. #\[serde(default)\] covers the missing-field case on inbound JSON (e.g. documents serialised before this field existed).
content	NodeContent	—	Node content — tagged enum, type-specific data only.
parent	number | null	null	Parent node index (null = root-level node).
children	Array<number>	/* serde(default) */	Child node indices in reading order.
contentLayer	ContentLayer	/* serde(default) */	Content layer classification. Always serialised — Kotlin-Android (and any other typed binding) treats the field as non-nullable, so omitting it from the JSON wire would break consumer deserialisation. #\[serde(default)\] covers the missing-field case on inbound JSON.
page	number | null	null	Page number where this node starts (1-indexed).
pageEnd	number | null	null	Page number where this node ends (for multi-page tables/sections).
bbox	BoundingBox | null	null	Bounding box in document coordinates.
annotations	Array<TextAnnotation>	/* serde(default) */	Inline annotations (formatting, links) on this node’s text content. Only meaningful for text-carrying nodes; empty for containers.
attributes	Record<string, string> | null	null	Format-specific key-value attributes. Extensible bag for miscellaneous data without a dedicated typed field: CSS classes, LaTeX environment names, Excel cell formulas, slide layout names, etc.

---

DocumentRelationship

A resolved relationship between two nodes in the document tree.

Field	Type	Default	Description
source	number	—	Source node index (the referencing node).
target	number	—	Target node index (the referenced node).
kind	RelationshipKind	—	Semantic kind of the relationship.

---

DocumentRevision

A single tracked change embedded in a document.

Populated by per-format extractors that understand change-tracking metadata (DOCX w:ins/w:del/w:rPrChange, ODT text:change-*, …). Every extractor defaults to ExtractedDocument.revisions = None until a format-specific implementation is added.

Field	Type	Default	Description
revisionId	string	—	Format-specific revision identifier. For DOCX this is the w:id attribute value on the change element (e.g. "42"). When the attribute is absent a synthetic fallback is generated ("docx-ins-0", "docx-del-3", …).
author	string | null	null	Display name of the author who made this change, when available.
timestamp	string | null	null	ISO-8601 timestamp of the change, when available. Stored as a plain string so this type remains FFI-friendly and unconditionally available without the chrono optional dep. DOCX populates this from the w:date attribute (e.g. "2024-03-15T10:30:00Z").
kind	RevisionKind	—	Semantic kind of this revision.
anchor	RevisionAnchor | null	null	Best-effort document location for this revision. Resolution is format-dependent and may be null when the location cannot be determined (e.g. changes inside table cells before table-cell anchor support is added).
delta	RevisionDelta	—	The content changes that make up this revision.

---

DocumentStructure

Top-level structured document representation.

A flat array of nodes with index-based parent/child references forming a tree. Root-level nodes have parent: None. Use body_roots() and furniture_roots() to iterate over top-level content by layer.

Validation

Call validate() after construction to verify all node indices are in bounds and parent-child relationships are bidirectionally consistent.

Field	Type	Default	Description
nodes	Array<DocumentNode>	\[\]	All nodes in document/reading order.
sourceFormat	string | null	null	Origin format identifier (e.g. “docx”, “pptx”, “html”, “pdf”). Allows renderers to apply format-aware heuristics when converting the document tree to output formats.
relationships	Array<DocumentRelationship>	\[\]	Resolved relationships between nodes (footnote refs, citations, anchor links, etc.). Populated during derivation from the internal document representation. Empty when no relationships are detected.
nodeTypes	Array<string>	\[\]	Sorted, deduplicated list of node type names present in this document. Each value is the snake_case node_type tag of the corresponding NodeContent variant (e.g. "paragraph", "heading", "table", …). Computed from nodes via DocumentStructure.finalize_node_types. Empty until that method is called (internal construction paths call it at the end of derivation).

Methods

finalizeNodeTypes()

Compute and populate the node_types field from the current nodes.

Call this after all nodes have been added to the structure. Internal construction paths (builder, derivation) call this automatically.

Signature:

finalizeNodeTypes(): void

Example:

instance.finalizeNodeTypes();

Returns: No return value.

isEmpty()

Check if the document structure is empty.

Signature:

isEmpty(): boolean

Example:

const result = instance.isEmpty();

Returns: boolean

default()

Signature:

static default(): DocumentStructure

Example:

const result = DocumentStructure.default();

Returns: DocumentStructure

---

DocumentSummary

Summary of an extracted document.

Field	Type	Default	Description
text	string	—	Summary text (plain prose).
strategy	SummaryStrategy	—	Strategy that produced this summary.
tokenCount	number | null	null	Approximate token count of the summary, when known.

---

DocxAppProperties

Application properties from docProps/app.xml for DOCX

Contains Word-specific document statistics and metadata.

Field	Type	Default	Description
application	string | null	null	Application name (e.g., “Microsoft Office Word”)
appVersion	string | null	null	Application version
template	string | null	null	Template filename
totalTime	number | null	null	Total editing time in minutes
pages	number | null	null	Number of pages
words	number | null	null	Number of words
characters	number | null	null	Number of characters (excluding spaces)
charactersWithSpaces	number | null	null	Number of characters (including spaces)
lines	number | null	null	Number of lines
paragraphs	number | null	null	Number of paragraphs
company	string | null	null	Company name
docSecurity	number | null	null	Document security level
scaleCrop	boolean | null	null	Scale crop flag
linksUpToDate	boolean | null	null	Links up to date flag
sharedDoc	boolean | null	null	Shared document flag
hyperlinksChanged	boolean | null	null	Hyperlinks changed flag

---

DocxMetadata

Word document metadata.

Extracted from DOCX files using shared Office Open XML metadata extraction. Integrates with office_metadata module for core/app/custom properties.

Field	Type	Default	Description
coreProperties	CoreProperties | null	null	Core properties from docProps/core.xml (Dublin Core metadata) Contains title, creator, subject, keywords, dates, etc. Shared format across DOCX/PPTX/XLSX documents.
appProperties	DocxAppProperties | null	null	Application properties from docProps/app.xml (Word-specific statistics) Contains word count, page count, paragraph count, editing time, etc. DOCX-specific variant of Office application properties.
customProperties	Record<string, unknown> | null	{}	Custom properties from docProps/custom.xml (user-defined properties) Contains key-value pairs defined by users or applications. Values can be strings, numbers, booleans, or dates.

---

Element

Semantic element extracted from document.

Represents a logical unit of content with semantic classification, unique identifier, and metadata for tracking origin and position.

Field	Type	Default	Description
elementType	ElementType	—	Semantic type of this element
text	string	—	Text content of the element
metadata	ElementMetadata	—	Metadata about the element

---

ElementMetadata

Metadata for a semantic element.

Field	Type	Default	Description
pageNumber	number | null	null	Page number (1-indexed)
filename	string | null	null	Source filename or document name
coordinates	BoundingBox | null	null	Bounding box coordinates if available
elementIndex	number | null	null	Position index in the element sequence
additional	Record<string, string>	—	Additional custom metadata

---

EmailAttachment

Email attachment representation.

Contains metadata and optionally the content of an email attachment.

Field	Type	Default	Description
name	string | null	null	Attachment name (from Content-Disposition header)
filename	string | null	null	Filename of the attachment
mimeType	string | null	null	MIME type of the attachment
size	number | null	null	Size in bytes
isImage	boolean	—	Whether this attachment is an image
data	Buffer | null	null	Attachment data (if extracted). Uses bytes.Bytes for cheap cloning of large buffers.

---

EmailConfig

Configuration for email extraction.

Field	Type	Default	Description
msgFallbackCodepage	number | null	null	Windows codepage number to use when an MSG file contains no codepage property. Defaults to null, which falls back to windows-1252. If an unrecognized or invalid codepage number is supplied (including 0), the behavior silently falls back to windows-1252 — the same as when the MSG file itself contains an unrecognized codepage. No error or warning is emitted. Users should verify output when supplying unusual values. Common values: - 1250: Central European (Polish, Czech, Hungarian, etc.) - 1251: Cyrillic (Russian, Ukrainian, Bulgarian, etc.) - 1252: Western European (default) - 1253: Greek - 1254: Turkish - 1255: Hebrew - 1256: Arabic - 932: Japanese (Shift-JIS) - 936: Simplified Chinese (GBK)

---

EmailExtractionResult

Email extraction result.

Complete representation of an extracted email message (.eml or .msg) including headers, body content, and attachments.

Field	Type	Default	Description
subject	string | null	null	Email subject line
fromEmail	string | null	null	Sender email address
toEmails	Array<string>	—	Primary recipient email addresses
ccEmails	Array<string>	—	CC recipient email addresses
bccEmails	Array<string>	—	BCC recipient email addresses
date	string | null	null	Email date/timestamp
messageId	string | null	null	Message-ID header value
plainText	string | null	null	Plain text version of the email body
htmlContent	string | null	null	HTML version of the email body
content	string	—	Cleaned/processed text content. Aliased as cleaned_text for back-compat.
attachments	Array<EmailAttachment>	—	List of email attachments
metadata	Record<string, string>	—	Additional email headers and metadata

---

EmailMetadata

Email metadata extracted from .eml and .msg files.

Includes sender/recipient information, message ID, and attachment list.

Field	Type	Default	Description
fromEmail	string | null	null	Sender’s email address
fromName	string | null	null	Sender’s display name
toEmails	Array<string>	\[\]	Primary recipients
ccEmails	Array<string>	\[\]	CC recipients
bccEmails	Array<string>	\[\]	BCC recipients
messageId	string | null	null	Message-ID header value
attachments	Array<string>	\[\]	List of attachment filenames

---

EmbeddedChanges

Changes to embedded archive children between two results.

Field	Type	Default	Description
added	Array<ArchiveEntry>	\[\]	Children present in b but not in a (matched by path).
removed	Array<ArchiveEntry>	\[\]	Children present in a but not in b (matched by path).
changed	Array<EmbeddedDiff>	\[\]	Children present in both but with differing content (matched by path). Each entry holds the diff of the nested ExtractedDocument.

---

EmbeddedDiff

Diff for a single embedded archive entry that appears in both results.

Field	Type	Default	Description
path	string	—	Archive-relative path identifying this entry.
diff	ExtractionDiff	—	The recursive diff of the entry’s extraction result.

---

EmbeddedFile

Embedded file descriptor extracted from the PDF name tree.

Field	Type	Default	Description
name	string	—	The filename as stored in the PDF name tree.
data	Buffer	—	Raw file bytes from the embedded stream (already decompressed by lopdf).
compressedSize	number	—	Compressed byte count of the original stream (before decompression). Used by callers to compute the decompression ratio and detect zip-bomb-style attacks that embed a tiny compressed stream expanding to gigabytes of data.
mimeType	string | null	null	MIME type if specified in the filespec, otherwise null.

---

EmbeddingBackend

Trait for in-process embedding backend plugins.

Async to match the convention used by other plugin hooks such as OcrBackend and PostProcessor. Host-language bridges (PyO3, napi-rs, Rustler, extendr, magnus, ext-php-rs, C FFI, etc.) wrap their synchronous host callables in spawn_blocking or the equivalent to satisfy the async signature.

Thread safety

Backends must be Send + Sync + 'static. They are stored in Arc<dyn EmbeddingBackend> and called concurrently from xberg’s chunking pipeline. If the backend’s underlying model isn’t thread-safe, the backend itself must serialize access internally (e.g. via Mutex<Inner>).

Contract

• embed(texts) MUST return exactly texts.len() vectors, each of length self.dimensions(). The dispatcher in crate.embeddings.embed_texts validates this before returning to downstream consumers; a non-conforming backend surfaces as a XbergError.Validation, not a panic.

• embed may be called from any thread. Its future must be Send (enforced by async_trait when #[async_trait] is used on non-WASM targets).

• dimensions() is called exactly once at registration, immediately after initialize() succeeds. The returned value is cached by the registry and used for all subsequent shape validation. Lazy-loading implementations can defer model loading into initialize() and report the real dimension afterwards. Later mutations of the backend’s reported dimension are not observed by xberg — implementations that need to change dimension must unregister and re-register.

• shutdown() (inherited from Plugin) may be invoked concurrently with an in-flight embed() call. Implementations must tolerate this — e.g. by letting in-flight calls finish using resources held via the Arc<dyn EmbeddingBackend> reference, and only releasing shared state that isn’t needed by embed.

Runtime

The synchronous embed_texts entry uses tokio.task.block_in_place to await the trait’s async embed, which requires a multi-thread tokio runtime. Callers running inside a current_thread runtime (e.g. #[tokio.test] without flavor = "multi_thread", or tokio.runtime.Builder.new_current_thread()) must use embed_texts_async instead, which awaits directly without block_in_place.

Methods

dimensions()

Embedding vector dimension. Must be > 0 and must match the length of every vector returned by embed.

Signature:

dimensions(): number

Example:

const result = instance.dimensions();

Returns: number

embed()

Embed a batch of texts, returning one vector per input in order.

Errors:

Implementations should return Plugin for backend-specific failures. The dispatcher layers its own validation (length, per-vector dimension) on top.

Signature:

embed(texts: Array<string>): Promise<Array<Array<number>>>

Example:

const result = await instance.embed([]);

Parameters:

Name	Type	Required	Description
texts	Array<string>	Yes	The texts

Returns: Array<Array<number>>

Errors: Throws Error with a descriptive message.

---

EmbeddingConfig

Embedding configuration for text chunks.

Configures embedding generation using ONNX models via the vendored embedding engine. Requires the embeddings feature to be enabled.

Field	Type	Default	Description
model	EmbeddingModelType	EmbeddingModelType.Preset	The embedding model to use (defaults to “gte-modernbert-base” preset if not specified)
normalize	boolean	true	Whether to normalize embedding vectors (recommended for cosine similarity)
batchSize	number	32	Batch size for embedding generation
showDownloadProgress	boolean	false	Show model download progress. When enabled, transfer progress for the model, tokenizer and config files is reported at info level on the xberg.model_download target while they download (#279). Covers both local backends (ONNX and static/model2vec). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by EmbeddingModelType.Llm and EmbeddingModelType.Plugin, which download no model.
cacheDir	string | null	null	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows HF_HUB_CACHE, HUGGINGFACE_HUB_CACHE, HF_HOME, XDG, and platform defaults. Prefer those environment variables when configuring the cache process-wide.
acceleration	AccelerationConfig | null	null	Hardware acceleration for the embedding ONNX model. When set, controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for inference. Defaults to null (auto-select per platform).
maxEmbedDurationSecs	number | null	60	Maximum wall-clock duration (in seconds) for a single embed() call when using EmbeddingModelType.Plugin. Applies only to the in-process plugin path — protects against hung host-language backends (e.g. a Python callback deadlocked on the GIL, a model stuck on CUDA OOM retries, etc.). On timeout, the dispatcher returns Plugin instead of blocking forever. null disables the timeout. The default (60 seconds) is conservative for common in-process inference; increase for large batches on slow hardware.
maxSequenceLength	number | null	null	Maximum number of tokens fed to the tokenizer before truncation when embedding a chunk with a local ONNX model (Preset/Custom). A chunk longer than this many tokens has its tail dropped before inference, so only the prefix contributes to the stored vector. null falls back to 512 (the historical default). The effective value is always capped at the model’s own model_max_length, so raising it past what the model supports has no effect — set it to match a long-context model (e.g. 8192 for Jina/Nomic) so long chunks embed in full. Ignored by the Llm and Plugin model types, which own their own tokenization.

Methods

default()

Signature:

static default(): EmbeddingConfig

Example:

const result = EmbeddingConfig.default();

Returns: EmbeddingConfig

---

Entity

A single named entity detected in the extracted text.

Field	Type	Default	Description
category	EntityCategory	—	Canonical category the entity belongs to (PERSON, ORG, LOCATION, etc.).
text	string	—	Raw mention text exactly as it appeared in the source.
start	number	—	Byte-offset span in ExtractedDocument.content where the mention starts.
end	number	—	Byte-offset span in ExtractedDocument.content where the mention ends (exclusive).
confidence	number | null	null	Backend-reported confidence in \[0.0, 1.0\]. null when the backend does not expose confidence scores.

---

EpubMetadata

EPUB metadata (Dublin Core extensions).

Field	Type	Default	Description
coverage	string | null	null	Dublin Core coverage field (geographic or temporal scope).
dcFormat	string | null	null	Dublin Core format field (media type of the resource).
relation	string | null	null	Dublin Core relation field (related resource identifier).
source	string | null	null	Dublin Core source field (origin resource identifier).
dcType	string | null	null	Dublin Core type field (nature or genre of the resource).
coverImage	string | null	null	Path or identifier of the cover image within the EPUB container.

---

ErrorMetadata

Error metadata (for batch operations).

Field	Type	Default	Description
errorType	string	—	Machine-readable error type identifier (e.g. “UnsupportedFormat”).
message	string	—	Human-readable error description.

---

ExcelMetadata

Excel/spreadsheet format metadata.

Identifies the document as a spreadsheet source via the FormatMetadata.Excel discriminant. Sheet count and sheet names are stored inside this struct.

Field	Type	Default	Description
sheetCount	number | null	null	Number of sheets in the workbook.
sheetNames	Array<string> | null	\[\]	Names of all sheets in the workbook.

---

ExcelSheet

Single Excel worksheet.

Represents one sheet from an Excel workbook with its content converted to Markdown format and dimensional statistics.

Field	Type	Default	Description
name	string	—	Sheet name as it appears in Excel
markdown	string	—	Sheet content converted to Markdown tables
rowCount	number	—	Number of rows
colCount	number	—	Number of columns
cellCount	number	—	Total number of non-empty cells
tableCells	Array<Array<string>> | null	null	Pre-extracted table cells (2D vector of cell values) Populated during markdown generation to avoid re-parsing markdown. None for empty sheets.

---

ExcelWorkbook

Excel workbook representation.

Contains all sheets from an Excel file (.xlsx, .xls, etc.) with extracted content and metadata.

Field	Type	Default	Description
sheets	Array<ExcelSheet>	—	All sheets in the workbook
metadata	Record<string, string>	—	Workbook-level metadata (author, creation date, etc.)
revisions	Array<DocumentRevision> | null	/* serde(default) */	Collaborative-edit revision headers from xl/revisions/revisionHeaders.xml. Populated for legacy shared-workbook .xlsx files that contain the xl/revisions/ directory. Each <header> element maps to one DocumentRevision { kind: FormatChange } carrying the header’s guid (→ revision_id), userName (→ author), and dateTime (→ timestamp). anchor and delta are null/empty for v1 (per-cell log parsing is a follow-up). null when xl/revisions/revisionHeaders.xml is absent.

---

ExtractInput

Unified extraction input for all public extraction entry points.

Field	Type	Default	Description
kind	ExtractInputKind	ExtractInputKind.Uri	Source kind. bytes requires bytes; uri requires uri.
bytes	Buffer | null	null	Raw bytes for kind = "bytes".
uri	string | null	null	Local path, file:// URI, or HTTP(S) URL for kind = "uri".
mimeType	string | null	null	MIME type hint.
filename	string | null	null	Filename hint used for MIME detection and metadata.
config	FileExtractionConfig | null	null	Per-input extraction overrides.

Methods

default()

Signature:

static default(): ExtractInput

Example:

const result = ExtractInput.default();

Returns: ExtractInput

fromBytes()

Build a bytes input with a MIME type and optional filename hint.

Signature:

static fromBytes(bytes: Buffer, mimeType: string, filename: string): ExtractInput

Example:

const result = ExtractInput.fromBytes(Buffer.from("data"), "value", "value");

Parameters:

Name	Type	Required	Description
bytes	Buffer	Yes	The bytes
mimeType	string	Yes	The mime type
filename	string | null	No	The filename

Returns: ExtractInput

fromUri()

Build a URI input from a local path, file:// URI, or HTTP(S) URL.

Signature:

static fromUri(uri: string): ExtractInput

Example:

const result = ExtractInput.fromUri("value");

Parameters:

Name	Type	Required	Description
uri	string	Yes	The uri

Returns: ExtractInput

---

ExtractedDocument

Document extracted by the core extraction pipeline.

extract and extract_batch return an ExtractionResult envelope whose results field contains these per-document payloads.

Field	Type	Default	Description
content	string	—	Plain-text representation of the extracted document content.
mimeType	string	—	MIME type of the source document (e.g. "application/pdf").
metadata	Metadata	—	Document-level metadata (author, title, dates, format-specific fields).
extractionMethod	ExtractionMethod | null	null	Extraction strategy used to produce the returned text. Populated when the extractor can reliably distinguish native text extraction, OCR-only extraction, or mixed native/OCR output.
tables	Array<Table>	\[\]	Tables extracted from the document, each with structured cell data.
counts	DocumentCounts	—	Cheap structural counts (pages, tables, images). Always populated by the extraction pipeline, even when the pages / images collections are null. See DocumentCounts.
detectedLanguages	Array<string> | null	\[\]	ISO 639-1 language codes detected in the document content.
detectedLanguageConfidences	Array<LanguageConfidence> | null	\[\]	Structured per-language detection results: confidence, document share, script, and reliability, alongside the ISO-code-only detected_languages (#261). One entry per language in detected_languages, in the same order. null under the same conditions as detected_languages: detection disabled, empty input text, or no language met the configured min_confidence.
chunks	Array<Chunk> | null	\[\]	Text chunks when chunking is enabled. When chunking configuration is provided, the content is split into overlapping chunks for efficient processing. Each chunk contains the text, optional embeddings (if enabled), and metadata about its position.
images	Array<ExtractedImage> | null	\[\]	Extracted images from the document. When image extraction is enabled via ImageExtractionConfig, this field contains all images found in the document with their raw data and metadata. Each image may optionally contain a nested ocr_result if OCR was performed.
pages	Array<PageContent> | null	\[\]	Per-page content when page extraction is enabled. When page extraction is configured, the document is split into per-page content with tables and images mapped to their respective pages.
elements	Array<Element> | null	\[\]	Semantic elements when element-based result format is enabled. When result_format is set to ElementBased, this field contains semantic elements with type classification, unique identifiers, and metadata for Unstructured-compatible element-based processing.
djotContent	DjotContent | null	null	Rich Djot content structure (when extracting Djot documents). When extracting Djot documents with structured extraction enabled, this field contains the full semantic structure including: - Block-level elements with nesting - Inline formatting with attributes - Links, images, footnotes - Math expressions - Complete attribute information The content field still contains plain text for backward compatibility. Always null for non-Djot documents.
ocrElements	Array<OcrElement> | null	\[\]	OCR elements with full spatial and confidence metadata. When OCR is performed with element extraction enabled, this field contains the structured representation of detected text including: - Bounding geometry (rectangles or quadrilaterals) - Confidence scores (detection and recognition) - Rotation information - Hierarchical relationships (Tesseract only) This field preserves all metadata that would otherwise be lost when converting to plain text or markdown output formats. Only populated when OcrElementConfig.include_elements is true.
document	DocumentStructure | null	null	Structured document tree (when document structure extraction is enabled). When include_document_structure is true in ExtractionConfig, this field contains the full hierarchical representation of the document including: - Heading-driven section nesting - Table grids with cell-level metadata - Content layer classification (body, header, footer, footnote) - Inline text annotations (formatting, links) - Bounding boxes and page numbers Independent of result_format — can be combined with Unified or ElementBased.
extractedKeywords	Array<Keyword> | null	\[\]	Extracted keywords when keyword extraction is enabled. When keyword extraction (RAKE or YAKE) is configured, this field contains the extracted keywords with scores, algorithm info, and position data. Previously stored in metadata.additional\["keywords"\].
qualityScore	number | null	null	Document quality score from quality analysis. A value between 0.0 and 1.0 indicating the overall text quality. Previously stored in metadata.additional\["quality_score"\].
processingWarnings	Array<ProcessingWarning>	\[\]	Non-fatal warnings collected during processing pipeline stages. Captures errors from optional pipeline features (embedding, chunking, language detection, output formatting) that don’t prevent extraction but may indicate degraded results. Previously stored as individual keys in metadata.additional.
annotations	Array<PdfAnnotation> | null	\[\]	PDF annotations extracted from the document. When annotation extraction is enabled via PdfConfig.extract_annotations, this field contains text notes, highlights, links, stamps, and other annotations found in PDF documents.
children	Array<ArchiveEntry> | null	\[\]	Nested extraction results from archive contents. When extracting archives, each processable file inside produces its own full extraction result. Set to null for non-archive formats. Use max_archive_depth in config to control recursion depth.
uris	Array<ExtractedUri> | null	\[\]	URIs/links discovered during document extraction. Contains hyperlinks, image references, citations, email addresses, and other URI-like references found in the document. Always extracted when present in the source document.
revisions	Array<DocumentRevision> | null	\[\]	Tracked changes embedded in the source document. Populated by per-format extractors that understand change-tracking metadata (DOCX w:ins/w:del/w:rPrChange, ODT text:change-*, …). Every extractor defaults to null until its format-specific implementation is added. Extractors that do populate this field follow the “accepted-changes” convention: inserted text is present in content, deleted text is absent — the revision list is the separate audit trail.
structuredOutput	unknown | null	null	Structured extraction output from LLM-based JSON schema extraction. When structured_extraction is configured in ExtractionConfig, the extracted document content is sent to a VLM with the provided JSON schema. The response is parsed and stored here as a JSON value matching the schema.
codeIntelligence	unknown | null	null	Code intelligence results from tree-sitter analysis. Populated when extracting source code files with the tree-sitter feature. Contains metrics, structural analysis, imports/exports, comments, docstrings, symbols, diagnostics, and optionally chunked code segments. Stored as an opaque JSON value so that all language bindings (Go, Java, C#, …) can deserialize it as a raw JSON object rather than a typed struct. The underlying type is tree_sitter_language_pack.ProcessResult.
llmUsage	Array<LlmUsage> | null	\[\]	LLM token usage and cost data for all LLM calls made during this extraction. Contains one entry per LLM call. Multiple entries are produced when VLM OCR, structured extraction, or LLM embeddings run during the same extraction. null when no LLM was used.
entities	Array<Entity> | null	\[\]	Named entities detected in content by the NER post-processor. null when no NER backend is configured. Populated by the xberg-gliner ONNX backend or the LLM-driven backend (see crates/xberg/src/text/ner/).
summary	DocumentSummary | null	null	Summary of content produced by the summarisation post-processor. null when summarisation is not configured. Populated by the TextRank extractive backend (deterministic, no external service) or by the liter-llm-driven abstractive backend.
extractionConfidence	ExtractionConfidence | null	null	Confidence score computed by the heuristics pipeline. Populated when the heuristics feature is enabled and confidence scoring has been performed. Combines text-coverage, OCR aggregate confidence, and schema-compliance into a single \[0, 1\] value. null when confidence scoring is not configured or the feature is absent.
translation	Translation | null	null	Translation of content produced by the translation post-processor. null when translation is not configured.
pageClassifications	Array<PageClassification> | null	\[\]	Per-page classifications produced by the page-classification post-processor. null when classification is not configured.
redactionReport	RedactionReport | null	null	Audit report of redactions applied by the redaction post-processor. The redaction processor rewrites content, formatted_content, every chunk’s text, and the textual fields of entities / summary / translation / page_classifications in place. This report describes what was found and how it was replaced. null when redaction is not configured.
formulas	Array<Formula>	\[\]	Mathematical formulas recognized in the document. Populated by the layout-guided formula pipeline when the layout-detection feature is enabled and the document contains regions classified as formulas. Empty otherwise.
formFields	Array<PdfFormField>	\[\]	Form fields extracted from a PDF’s AcroForm or XFA structure. Populated by the PDF extractor when PdfConfig.extract_form_fields is enabled (default) and the document is a fillable form. Empty otherwise.

---

ExtractedImage

Extracted image from a document.

Contains raw image data, metadata, and optional nested OCR results. Raw bytes allow cross-language compatibility - users can convert to PIL.Image (Python), Sharp (Node.js), or other formats as needed.

Field	Type	Default	Description
data	Buffer	—	Raw image data (PNG, JPEG, WebP, etc. bytes). Uses bytes.Bytes for cheap cloning of large buffers.
format	string	—	Image format (e.g., “jpeg”, “png”, “webp”) Uses Cow<’static, str> to avoid allocation for static literals.
imageIndex	number	—	Zero-indexed position of this image in the document/page
pageNumber	number | null	null	Page/slide number where image was found (1-indexed)
width	number | null	null	Image width in pixels
height	number | null	null	Image height in pixels
colorspace	string | null	null	Colorspace information (e.g., “RGB”, “CMYK”, “Gray”)
bitsPerComponent	number | null	null	Bits per color component (e.g., 8, 16)
isMask	boolean	—	Whether this image is a mask image
description	string | null	null	Optional description of the image
ocrResult	ExtractedDocument | null	null	Nested OCR extraction result (if image was OCRed) When OCR is performed on this image, the result is embedded here rather than in a separate collection, making the relationship explicit.
boundingBox	BoundingBox | null	null	Bounding box of the image on the page (PDF coordinates: x0=left, y0=bottom, x1=right, y1=top). Only populated for PDF-extracted images when position data is available from the PDF extractor.
sourcePath	string | null	null	Original source path of the image within the document archive (e.g., “media/image1.png” in DOCX). Used for rendering image references when the binary data is not extracted.
imageKind	ImageKind | null	null	Heuristic classification of what this image likely depicts. null if classification was disabled or inconclusive.
kindConfidence	number | null	null	Confidence score for image_kind, in the range 0.0 to 1.0.
clusterId	number | null	null	Identifier shared across images that form a single logical figure (e.g. all raster tiles of one technical drawing). null for singletons.
caption	string | null	null	VLM-generated caption describing the image, when captioning is configured. Populated by the captioning post-processor (crates/xberg/src/plugins/processor/builtin/captioning.rs), which routes each image through crate.llm.region_extractor.extract_region_with_vlm in caption mode. null when captioning is disabled or the VLM declined to caption.
qrCodes	Array<QrCode> | null	\[\]	QR codes decoded from this image, when QR detection is enabled. Populated by the QR post-processor (crates/xberg/src/extractors/qr.rs) via the pure-Rust rqrr decoder. null when QR detection is disabled; an empty Some(\[\]) when detection ran but found nothing.
dataBase64	string | null	null	Base64-encoded copy of data; populated when ImageExtractionConfig.include_data_base64 is true. Omitted from JSON by default; use instead of data in JSON-only clients.

---

ExtractedUri

A URI extracted from a document.

Represents any link, reference, or resource pointer found during extraction. The kind field classifies the URI semantically, while label carries optional human-readable display text.

Field	Type	Default	Description
url	string	—	The URL or path string.
label	string | null	null	Optional display text / label for the link.
page	number | null	null	Optional page number where the URI was found (1-indexed).
kind	UriKind	—	Semantic classification of the URI.

---

ExtractionConfidence

Combined confidence on [0, 1].

When OCR did not run, the ocr_aggregate weight folds into text_coverage so the weighted sum still totals 1.0.

Field	Type	Default	Description
textCoverage	number	—	Fraction of pages with a usable text layer.
ocrAggregate	number | null	null	Mean OCR per-element recognition confidence when OCR ran; null when it did not.
schemaCompliance	SchemaCompliance	—	Whether the merged output validates against the preset schema.
combined	number	—	Weighted blend in \[0, 1\]. The value compared against the fallback threshold.

---

ExtractionConfig

Main extraction configuration.

This struct contains all configuration options for the extraction process. It can be loaded from TOML, YAML, or JSON files, or created programmatically.

Field	Type	Default	Description
useCache	boolean	true	Enable caching of extraction results
enableQualityProcessing	boolean	true	Enable quality post-processing
ocr	OcrConfig | null	null	OCR configuration. null does not run OCR for documents that already have usable text. Under OcrStrategy.Auto, a PDF with no text layer at all (a scan) is still routed to OCR with default settings so it is not returned empty (#1338). Set Self.disable_ocr to hard-disable OCR regardless of the detected content.
forceOcr	boolean	false	Force OCR even for searchable PDFs
ocrStrategy	OcrStrategy	OcrStrategy.Auto	Which pages get OCR’d when neither force_ocr nor force_ocr_pages applies. Defaults to OcrStrategy.Auto, which OCRs only pages whose native text fails a quality check. Only applies to PDF documents. Cannot be OcrStrategy.ScannedPages while disable_ocr is true.
forceOcrPages	Array<number> | null	null	Force OCR on specific pages only (1-indexed page numbers, must be >= 1). When set, only the listed pages are OCR’d regardless of text layer quality. Unlisted pages use native text extraction. Ignored when force_ocr is true. Only applies to PDF documents. Duplicates are automatically deduplicated. An ocr config is recommended for backend/language selection; defaults are used if absent.
disableOcr	boolean	false	Disable OCR entirely, even for images. When true, OCR is skipped for all document types. Images return metadata only (dimensions, format, EXIF) without text extraction. PDFs use only native text extraction without OCR fallback. Cannot be true simultaneously with force_ocr.
chunking	ChunkingConfig | null	null	Text chunking configuration (None = chunking disabled)
contentFilter	ContentFilterConfig | null	null	Content filtering configuration (None = use extractor defaults). Controls whether document “furniture” (headers, footers, watermarks, repeating text) is included in or stripped from extraction results. See ContentFilterConfig for per-field documentation.
images	ImageExtractionConfig | null	null	Image extraction configuration (None = no image extraction)
pdfOptions	PdfConfig | null	null	PDF-specific options (None = use defaults)
tokenReduction	TokenReductionOptions | null	null	Token reduction configuration (None = no token reduction)
languageDetection	LanguageDetectionConfig | null	null	Language detection configuration (None = no language detection)
pages	PageConfig | null	null	Page extraction configuration (None = no page tracking)
keywords	KeywordConfig | null	null	Keyword extraction configuration (None = no keyword extraction)
postprocessor	PostProcessorConfig | null	null	Post-processor configuration (None = use defaults)
htmlOptions	ConversionOptions | null	null	HTML to Markdown conversion options (None = use defaults) Configure how HTML documents are converted to Markdown, including heading styles, list formatting, code block styles, and preprocessing options.
htmlOutput	HtmlOutputConfig | null	null	Styled HTML output configuration. When set alongside output_format = OutputFormat.Html, the extraction pipeline uses StyledHtmlRenderer which emits stable kb-* CSS class hooks on every structural element and optionally embeds theme CSS or user-supplied CSS in a <style> block. When null, the existing plain comrak-based HTML renderer is used.
extractionTimeoutSecs	number | null	null	Default per-file timeout in seconds for batch extraction. When set, each file in a batch will be canceled after this duration unless overridden by FileExtractionConfig.timeout_secs. Defaults to Some(600) (10 minutes) to prevent pathological files (e.g. deeply nested archives, documents with millions of cells) from running indefinitely and exhausting caller resources, while still giving slow paths (VLM-based OCR, large scanned documents) enough headroom to finish. Set to null to disable the timeout for trusted input or long-running workloads.
maxConcurrentExtractions	number | null	null	Maximum concurrent document extractions in batch operations. This is a ceiling within the configured total thread budget, not an independent pool size. When unset, the scheduler derives document and per-document concurrency from ConcurrencyConfig.max_threads.
resultFormat	ResultFormat	ResultFormat.Unified	Result structure format Controls whether results are returned in unified format (default) with all content in the content field, or element-based format with semantic elements (for Unstructured-compatible output).
securityLimits	SecurityLimits | null	null	Security limits for archive extraction. Controls maximum archive size, compression ratio, file count, and other security thresholds to prevent decompression bomb attacks. Also caps nesting depth, iteration count, entity / token length, total content size, and table cell count for every extraction path that ingests user-controlled bytes. When null, default limits are used.
maxEmbeddedFileBytes	number | null	null	Maximum uncompressed size in bytes for a single embedded file before recursive extraction is attempted (default: 50 MiB). Applies to embedded objects inside OOXML containers (DOCX, PPTX) and to email attachments processed via recursive extraction. Files that exceed this limit are skipped with a ProcessingWarning rather than passed to the extraction pipeline, preventing a single oversized embedded object from consuming unbounded memory or time. Set to null to disable the per-embedded-file cap (falls back to security_limits.max_archive_size as the only guard).
outputFormat	OutputFormat	OutputFormat.Plain	Content text format (default: Plain). Controls the format of the extracted content: - Plain: Raw extracted text (default) - Markdown: Markdown formatted output - Djot: Djot markup format (requires djot feature) - Html: HTML formatted output When set to a structured format, extraction results will include formatted output. The formatted_content field may be populated when format conversion is applied.
escapeMarkdown	boolean	true	Escape Markdown special characters in rendered prose (default: true). When output_format is Markdown or Djot, the renderer backslash-escapes CommonMark-significant leading characters (e.g. -, #) so that literal text such as #06-18 or - clause round-trips safely through a CommonMark parser instead of being reinterpreted as a heading or list marker. Table cell text is never escaped, so escaped prose can look inconsistent with table cells containing the same characters. Set this to false to disable prose escaping and make content, pages\[\].content, and chunks\[\].content read identically to table cell text — useful for LLM prompts or search indexing where CommonMark round-tripping does not matter. Defaults to true to preserve existing behavior.
tableAnchors	boolean	false	Emit an opt-in anchor marker before each table’s rendered Markdown block (default: false). When output_format is Markdown (or Djot) and this is true, the renderer inserts a \[TABLE:{table_id}\] marker immediately before each table’s Markdown in content, pages\[\].content, and chunks\[\].content, where table_id matches the corresponding entry’s table_id. This lets a consumer reconcile a rendered Markdown table block with its structured tables\[\] entry. Defaults to false so existing output is byte-identical unless explicitly enabled.
jupyterCellRendering	JupyterCellRendering	JupyterCellRendering.Both	Controls how Jupyter notebook (.ipynb) code cells are rendered. - Both (default): code source plus the notebook’s saved outputs - Source: only the code source (fenced code blocks) - Outputs: only the saved outputs Cells are never executed; Outputs/Both surface only outputs already stored in the notebook.
layout	LayoutDetectionConfig | null	null	Layout detection configuration (None = layout detection disabled). When set, PDF pages and images are analyzed for document structure (headings, code, formulas, tables, figures, etc.) using RT-DETR models via ONNX Runtime. For PDFs, layout hints override paragraph classification in the markdown pipeline. For images, per-region OCR is performed with markdown formatting based on detected layout classes. Requires the layout-detection feature to run inference; the field is present whenever the layout-types feature is active (which includes layout-detection as well as the no-ORT target groups).
transcription	TranscriptionConfig | null	null	Transcription (speech-to-text) configuration for audio/video files. When set and enabled, files with audio/video MIME types (mp3, mp4, m4a, wav, webm, etc.) are routed to the Whisper-based transcription pipeline. The actual heavy dependencies are only active under the transcription feature; the field is visible under transcription-types (including on WASM and Android targets that use the no-ORT preset). Default: null (transcription disabled). This is an additive, non-breaking change.
useLayoutForMarkdown	boolean	false	Run layout detection on the non-OCR PDF markdown path. When true and layout is Some(_), layout regions inform reading order, region grouping, and table detection while native font/tag semantics remain authoritative for headings, lists, code, and formulas. OCR layout classification is unchanged. This improves structural output at the cost of inference latency (~150-300ms/page CPU, ~20-50ms/page GPU). Default: false. Requires the layout-detection feature.
includeDocumentStructure	boolean	false	Enable structured document tree output. When true, populates the document field on ExtractedDocument with a hierarchical DocumentStructure containing heading-driven section nesting, table grids, content layer classification, and inline annotations. Independent of result_format — can be combined with Unified or ElementBased.
acceleration	AccelerationConfig | null	null	Hardware acceleration configuration for ONNX Runtime models. Controls execution provider selection for layout detection and embedding models. When null, uses platform defaults (CoreML on macOS, CUDA on Linux, CPU on Windows).
cacheNamespace	string | null	null	Cache namespace for tenant isolation. When set, cache entries are stored under {cache_dir}/{namespace}/. Must be alphanumeric, hyphens, or underscores only (max 64 chars). Different namespaces have isolated cache spaces on the same filesystem.
cacheTtlSecs	number | null	null	Per-request cache TTL in seconds. Overrides the global max_age_days for this specific extraction. When 0, caching is completely skipped (no read or write). When null, the global TTL applies.
email	EmailConfig | null	null	Email extraction configuration (None = use defaults). Currently supports configuring the fallback codepage for MSG files that do not specify one. See EmailConfig for details.
csv	CsvConfig | null	null	CSV/TSV extraction configuration (None = use defaults). Lets callers set an explicit delimiter and declare comment-line prefixes to skip, instead of relying solely on delimiter auto-detection. See CsvConfig for details.
url	UrlExtractionConfig	—	URL ingestion and crawl configuration.
maxArchiveDepth	number	—	Maximum recursion depth for archive extraction (default: 3). Set to 0 to disable recursive extraction (legacy behavior).
treeSitter	TreeSitterConfig | null	null	Tree-sitter language pack configuration (None = tree-sitter disabled). When set, enables code file extraction using tree-sitter parsers. Controls grammar download behavior and code analysis options.
structuredExtraction	StructuredExtractionConfig | null	null	Structured extraction via LLM (None = disabled). When set, the extracted document content is sent to an LLM with the provided JSON schema. The structured response is stored in ExtractedDocument.structured_output.
ner	NerConfig | null	null	Named-entity recognition configuration. When set, the NER post-processor runs at the Middle stage and populates ExtractedDocument.entities.
redaction	RedactionConfig | null	null	Redaction / anonymisation configuration. When set, the redaction post-processor runs at the Late stage and rewrites every textual field in ExtractedDocument, emitting an audit trail in ExtractedDocument.redaction_report.
summarization	SummarizationConfig | null	null	Summarisation configuration. When set, the summarisation post-processor runs at the Middle stage and populates ExtractedDocument.summary.
translation	TranslationConfig | null	null	Translation configuration. When set, the translation post-processor runs at the Middle stage and populates ExtractedDocument.translation.
pageClassification	PageClassificationConfig | null	null	Per-page classification configuration. When set, the classification post-processor runs at the Middle stage and populates ExtractedDocument.page_classifications.
chunkClassification	ChunkClassificationConfig | null	null	Per-chunk multi-label classification configuration. When set, the chunk-classification post-processor runs at the Middle stage (after chunking) and populates ChunkMetadata.classifications on every chunk.
captioning	CaptioningConfig | null	null	VLM captioning configuration for extracted images. When set, the captioning post-processor runs at the Middle stage and writes a caption into each ExtractedImage.caption.
qrCodes	boolean | null	null	Enable QR-code detection in extracted images. When true, the QR post-processor runs at the Middle stage and populates ExtractedImage.qr_codes.

Methods

default()

Signature:

static default(): ExtractionConfig

Example:

const result = ExtractionConfig.default();

Returns: ExtractionConfig

validate()

Validate the configuration, returning an error if any settings are invalid.

Checks:

• ocr: backend name, VLM backend/model requirements, language codes, and the vlm_fallback quality threshold (see OcrConfig.validate).

• chunking: max_characters is non-zero and overlap is smaller than it.

• token_reduction: mode is one of the recognized reduction levels.

• images: target_dpi, min_dpi, and max_dpi are all positive and within the supported range.

• language_detection: min_confidence is a [0.0, 1.0] value.

• csv: delimiter, when set, is exactly one ASCII character.

Called automatically when a config is loaded from a file (ExtractionConfig.from_file and friends) or built from a JSON override (crate.core.config.merge.merge_config_json). A config assembled directly through the typed Rust API or an FFI builder is not automatically validated — call this method explicitly before use in that case.

Errors:

Returns XbergError.Validation describing the first invalid setting found.

Signature:

validate(): void

Example:

instance.validate();

Returns: No return value.

Errors: Throws Error with a descriptive message.

needsImageData()

Check if image processing is needed by examining OCR and image extraction settings.

Returns true if either OCR is enabled or image extraction is configured, indicating that image decompression and processing should occur. Returns false if both are disabled, allowing optimization to skip unnecessary image decompression for text-only extraction workflows.

Optimization Impact

For text-only extractions (no OCR, no image extraction), skipping image decompression can improve CPU utilization by 5-10% by avoiding wasteful image I/O and processing when results won’t be used. Returns true when image binary data should be extracted.

True when config.images.extract_images is set, captioning is configured, or QR-code detection is enabled. Captioning and QR-code detection both require image bytes regardless of whether the caller also requested image extraction.

Signature:

needsImageData(): boolean

Example:

const result = instance.needsImageData();

Returns: boolean

needsImageProcessing()

Returns true when any image processing is needed during extraction.

Optimization Impact

For text-only extractions (no OCR, no image extraction, no captioning), skipping image decompression can improve CPU utilization by 5-10% by avoiding wasteful image I/O and processing when results won’t be used.

Signature:

needsImageProcessing(): boolean

Example:

const result = instance.needsImageProcessing();

Returns: boolean

---

ExtractionDiff

The complete diff between two ExtractedDocument values.

Field	Type	Default	Description
contentDiff	Array<DiffHunk>	\[\]	Unified-diff hunks for the content field. Empty when the content is identical.
tablesAdded	Array<Table>	\[\]	Tables present in b but not in a (by index position, excess right-side tables).
tablesRemoved	Array<Table>	\[\]	Tables present in a but not in b (by index position, excess left-side tables).
tablesChanged	Array<TableDiff>	\[\]	Cell-level changes for table pairs that share the same index and dimensions.
metadataChanged	unknown	—	Metadata difference, encoded as a JSON object with three top-level keys: added (keys present in b but not a), removed (keys present in a but not b), and changed (keys whose values differ — each entry is { "from": <value-in-a>, "to": <value-in-b> }). This is NOT RFC 6902 JSON Patch — we deliberately chose a flatter shape to avoid pulling in a json-patch crate. If you need RFC 6902 semantics (with JSON Pointer paths) feed a.metadata and b.metadata to your preferred json-patch impl directly.
embeddedChanges	EmbeddedChanges	—	Changes to embedded archive children.

---

ExtractionErrorItem

Non-fatal per-input extraction error captured by ExtractionResult.

Field	Type	Default	Description
index	number	—	Input index in the original request.
code	number	—	Stable numeric error code.
errorType	string	—	Stable snake_case error kind.
source	string	—	Best-effort source identifier.
message	string	—	Error message.

---

ExtractionResult

Unified extraction result envelope.

Field	Type	Default	Description
results	Array<ExtractedDocument>	\[\]	Extracted documents in discovery order.
errors	Array<ExtractionErrorItem>	\[\]	Non-fatal per-input errors.
summary	ExtractionSummary	—	Aggregate counts for the operation.
crawlFinalUrls	Array<string>	\[\]	Final URLs reached after redirects during URL ingestion.
crawlRedirectCount	number	—	Total redirects followed while fetching or crawling URLs.
crawlUniqueNormalizedUrls	Array<string>	\[\]	Unique normalized URLs discovered by crawls.

Methods

single()

Build an output containing one successful result.

Signature:

static single(result: ExtractedDocument): ExtractionResult

Example:

const result = ExtractionResult.single(new ExtractedDocument());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document

Returns: ExtractionResult

---

ExtractionSummary

Summary for a unified extraction call.

Field	Type	Default	Description
inputs	number	—	Number of inputs submitted by the caller.
results	number	—	Number of extraction results produced.
errors	number	—	Number of per-input errors.
remoteUrls	number	—	Number of URI inputs that resolved to remote HTTP(S) URLs.
pagesCrawled	number	—	Number of HTML pages crawled or scraped.
documentsDownloaded	number	—	Number of downloaded non-HTML documents extracted from URLs.

---

FictionBookMetadata

FictionBook (FB2) metadata.

Field	Type	Default	Description
genres	Array<string>	\[\]	Genre tags as declared in the FB2 <genre> elements.
sequences	Array<string>	\[\]	Book series (sequence) names, if any.
annotation	string | null	null	Short annotation / summary from the FB2 <annotation> element.

---

FileExtractionConfig

Per-file extraction configuration overrides for batch processing.

All fields are Option<T> — null means “use the batch-level default.” This type is used by config and extract_batch to allow heterogeneous extraction settings within a single batch.

Excluded Fields

The following ExtractionConfig fields are batch-level only and cannot be overridden per file:

• max_concurrent_extractions — controls batch parallelism
• use_cache — global caching policy
• acceleration — shared ONNX execution provider
• security_limits — global archive security policy

Field	Type	Default	Description
enableQualityProcessing	boolean | null	null	Override quality post-processing for this file.
ocr	OcrConfig | null	null	Override OCR configuration for this file (None in the Option = use batch default).
forceOcr	boolean | null	null	Override force OCR for this file.
ocrStrategy	OcrStrategy | null	null	Override the OCR page-selection strategy for this file.
forceOcrPages	Array<number> | null	\[\]	Override force OCR pages for this file (1-indexed page numbers).
disableOcr	boolean | null	null	Override disable OCR for this file.
chunking	ChunkingConfig | null	null	Override chunking configuration for this file.
contentFilter	ContentFilterConfig | null	null	Override content filtering configuration for this file.
images	ImageExtractionConfig | null	null	Override image extraction configuration for this file.
pdfOptions	PdfConfig | null	null	Override PDF options for this file.
tokenReduction	TokenReductionOptions | null	null	Override token reduction for this file.
languageDetection	LanguageDetectionConfig | null	null	Override language detection for this file.
pages	PageConfig | null	null	Override page extraction for this file.
keywords	KeywordConfig | null	null	Override keyword extraction for this file.
postprocessor	PostProcessorConfig | null	null	Override post-processor for this file.
htmlOutput	HtmlOutputConfig | null	null	Override styled HTML output configuration for this file.
resultFormat	ResultFormat | null	null	Override result format for this file.
outputFormat	OutputFormat | null	null	Override output content format for this file.
includeDocumentStructure	boolean | null	null	Override document structure output for this file.
layout	LayoutDetectionConfig | null	null	Override layout detection for this file.
transcription	TranscriptionConfig | null	null	Transcription configuration (see ExtractionConfig for docs).
timeoutSecs	number | null	null	Override per-file extraction timeout in seconds. When set, the extraction for this file will be canceled after the specified duration. A timed-out file produces an error result without affecting other files in the batch.
treeSitter	TreeSitterConfig | null	null	Override tree-sitter configuration for this file.
structuredExtraction	StructuredExtractionConfig | null	null	Override structured extraction configuration for this file. When set, enables LLM-based structured extraction with a JSON schema for this specific file. The extracted content is sent to a VLM/LLM and the response is parsed according to the provided schema.
url	UrlExtractionConfig | null	null	Override URL ingestion and crawl configuration for this file.
ner	NerConfig | null	null	Override named-entity recognition configuration for this file.
redaction	RedactionConfig | null	null	Override redaction configuration for this file.
summarization	SummarizationConfig | null	null	Override summarization configuration for this file.
translation	TranslationConfig | null	null	Override translation configuration for this file.
pageClassification	PageClassificationConfig | null	null	Override per-page classification configuration for this file.
chunkClassification	ChunkClassificationConfig | null	null	Override per-chunk classification configuration for this file.
captioning	CaptioningConfig | null	null	Override VLM captioning configuration for this file.
qrCodes	boolean | null	null	Override QR-code detection for this file.

---

Footnote

Footnote in Djot.

Field	Type	Default	Description
label	string	—	Footnote label
content	Array<FormattedBlock>	—	Footnote content blocks

---

FootnoteAnchor

A footnote anchor reference in markdown text.

Represents a [^label] use-site (not a definition).

Field	Type	Default	Description
label	string	—	The label of the footnote reference (e.g., “1” in \[^1\]).
offset	number	—	Byte offset of the anchor in the markdown text.

---

FootnoteConfig

Configuration for markdown footnote and citation parsing.

Field	Type	Default	Description
parseCitations	boolean	true	Whether to parse the structured citation block (default: true). When enabled, the parser will look for and extract citations from the block after --- + <!-- citations ... -->.

Methods

default()

Signature:

static default(): FootnoteConfig

Example:

const result = FootnoteConfig.default();

Returns: FootnoteConfig

withParseCitations()

Set whether to parse the citation block.

Signature:

withParseCitations(enabled: boolean): FootnoteConfig

Example:

const result = instance.withParseCitations(true);

Parameters:

Name	Type	Required	Description
enabled	boolean	Yes	The enabled

Returns: FootnoteConfig

---

FootnoteDefinition

A footnote definition from markdown text.

Represents [^label]: content declarations (including multi-line continuations).

Field	Type	Default	Description
label	string	—	The label of the footnote (e.g., “1” in \[^1\]: ...).
content	string	—	The full content of the footnote definition.
offset	number	—	Byte offset of the definition line in the markdown text.

---

FormattedBlock

Block-level element in a Djot document.

Represents structural elements like headings, paragraphs, lists, code blocks, etc.

Field	Type	Default	Description
blockType	BlockType	—	Type of block element
level	number | null	null	Heading level (1-6) for headings, or nesting level for lists
inlineContent	Array<InlineElement>	—	Inline content within the block
language	string | null	null	Language identifier for code blocks
code	string | null	null	Raw code content for code blocks
children	Array<FormattedBlock>	/* serde(default) */	Nested blocks for containers (blockquotes, list items, divs)

---

Formula

A mathematical formula detected and recognized in a document.

Populated by the layout-guided formula pipeline: regions classified as LayoutClass.Formula are routed to the formula OCR task, which returns the LaTeX source for the region. The field is always present on ExtractedDocument but only populated when the layout-detection feature is active and the document contains formula regions.

Field	Type	Default	Description
latex	string	—	LaTeX source of the recognized formula, without surrounding $$ delimiters. This field contains the raw LaTeX code as produced by the OCR backend. To render the formula in Markdown or other formats, wrap with $$..$$ delimiters as needed.
bbox	BoundingBox	—	Bounding box of the formula region on its page, in rendered-image pixel coordinates. The coordinates are in the space of the OCR-rendered page image at the OCR DPI (typically 300 DPI). These coordinates are NOT comparable to bounding boxes from native PDF text extraction, which use PDF point coordinates.
page	number	—	1-indexed page number the formula appears on in the document. This is set by the extraction pipeline based on which page the formula was found on.

---

GridCell

Individual grid cell with position and span metadata.

Field	Type	Default	Description
content	string	—	Cell text content.
row	number	—	Zero-indexed row position.
col	number	—	Zero-indexed column position.
rowSpan	number	serde(default = "default_span")	Number of rows this cell spans.
colSpan	number	serde(default = "default_span")	Number of columns this cell spans.
isHeader	boolean	/* serde(default) */	Whether this is a header cell.
bbox	BoundingBox | null	null	Bounding box for this cell (if available).

---

HeaderMetadata

Header/heading element metadata.

Field	Type	Default	Description
level	number	—	Header level: 1 (h1) through 6 (h6)
text	string	—	Normalized text content of the header
id	string | null	null	HTML id attribute if present
depth	number	—	Document tree depth at the header element
htmlOffset	number	—	Byte offset in original HTML document

---

HeadingContext

Heading context for a chunk within a Markdown document.

Contains the heading hierarchy from document root to this chunk’s section.

Field	Type	Default	Description
headings	Array<HeadingLevel>	—	The heading hierarchy from document root to this chunk’s section. Index 0 is the outermost (h1), last element is the most specific.

---

HeadingLevel

A single heading in the hierarchy.

Field	Type	Default	Description
level	number	—	Heading depth (1 = h1, 2 = h2, etc.)
text	string	—	The text content of the heading.

---

HeuristicsConfig

Configuration for document chunking and analysis heuristics.

Every threshold is a public field so callers can override any subset via struct-update syntax: HeuristicsConfig { text_layer_threshold: 0.5, ..the default constructor }.

Field	Type	Default	Description
enablePdfTextHeuristics	boolean	true	Enable PDF text-layer detection heuristics. When true, PDFs with a substantial text layer will skip chunking. Default: true.
textLayerThreshold	number	0.7	Minimum fraction of pages that must have text to skip chunking. Range 0.0..=1.0. Default: 0.7 (70 % of pages).
fileSizeThresholdBytes	number	10485760	File size threshold in bytes for considering chunking. Files smaller than this are processed without chunking. Default: 10 MiB (10 × 1 024 × 1 024).
pageCountThreshold	number	50	Page count threshold for considering chunking. Documents with fewer pages are processed without chunking. Default: 50.
targetPagesPerChunk	number	10	Target number of pages per chunk for optimal parallel processing. Default: 10.
maxPagesPerChunk	number	25	Hard cap on pages per chunk. No chunk will exceed this limit. Must be ≥ target_pages_per_chunk. Default: 25.
diskProcessingThresholdBytes	number	52428800	File size threshold for disk-based processing. Files larger than this are buffered to disk to prevent OOM. Default: 50 MiB (50 × 1 024 × 1 024).
minCharsPerPage	number	50	Minimum characters per page to consider a page as having text. Default: 50.
maxXlsxSheetCount	number	200	Maximum sheet count allowed in an XLSX workbook. Workbooks beyond this are rejected pre-extraction to avoid OOM / abusive billing inflation. Default: 200.
maxXlsxWorkbookCells	number	5000000	Maximum cell count (sheets × rows × columns approximation) in an XLSX workbook. Default: 5 000 000 (≈ 200 sheets × 25 k cells).
maxPptxEmbeddedCount	number	50	Maximum number of OLE-embedded objects extractable from a single PPTX or DOCX. Protects against zip-bomb-style nested-document abuse. Default: 50.

Methods

default()

Signature:

static default(): HeuristicsConfig

Example:

const result = HeuristicsConfig.default();

Returns: HeuristicsConfig

validate()

Validate the configuration.

Errors:

Returns HeuristicsError.ConfigError when:

• target_pages_per_chunk is 0
• max_pages_per_chunk < target_pages_per_chunk
• file_size_threshold_bytes is 0

Signature:

validate(): void

Example:

instance.validate();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

HierarchicalBlock

A text block with hierarchy level assignment.

Represents a block of text with semantic heading information extracted from font size clustering and hierarchical analysis.

Field	Type	Default	Description
text	string	—	The text content of this block
fontSize	number	—	The font size of the text in this block
level	string	—	The hierarchy level of this block (H1-H6 or Body) Levels correspond to HTML heading tags: - “h1”: Top-level heading - “h2”: Secondary heading - “h3”: Tertiary heading - “h4”: Quaternary heading - “h5”: Quinary heading - “h6”: Senary heading - “body”: Body text (no heading level)

---

HierarchyConfig

Hierarchy extraction configuration for PDF text structure analysis.

Enables extraction of document hierarchy levels (H1-H6) based on font size clustering and semantic analysis. When enabled, hierarchical blocks are included in page content.

Field	Type	Default	Description
enabled	boolean	true	Enable hierarchy extraction
kClusters	number	3	Number of font size clusters to use for hierarchy levels (1-7) Default: 6, which provides H1-H6 heading levels with body text. Larger values create more fine-grained hierarchy levels.
includeBbox	boolean	true	Include bounding box information in hierarchy blocks

Methods

default()

Signature:

static default(): HierarchyConfig

Example:

const result = HierarchyConfig.default();

Returns: HierarchyConfig

---

HtmlMetadata

HTML metadata extracted from HTML documents.

Includes document-level metadata, Open Graph data, Twitter Card metadata, and extracted structural elements (headers, links, images, structured data).

Field	Type	Default	Description
title	string | null	null	Document title from <title> tag
description	string | null	null	Document description from <meta name="description"> tag
keywords	Array<string>	\[\]	Document keywords from <meta name="keywords"> tag, split on commas
author	string | null	null	Document author from <meta name="author"> tag
canonicalUrl	string | null	null	Canonical URL from <link rel="canonical"> tag
baseHref	string | null	null	Base URL from <base href=""> tag for resolving relative URLs
language	string | null	null	Document language from lang attribute
textDirection	TextDirection | null	null	Document text direction from dir attribute
openGraph	Record<string, string>	{}	Open Graph metadata (og:* properties) for social media Keys like “title”, “description”, “image”, “url”, etc.
twitterCard	Record<string, string>	{}	Twitter Card metadata (twitter:* properties) Keys like “card”, “site”, “creator”, “title”, “description”, “image”, etc.
metaTags	Record<string, string>	{}	Additional meta tags not covered by specific fields Keys are meta name/property attributes, values are content
headers	Array<HeaderMetadata>	\[\]	Extracted header elements with hierarchy
links	Array<LinkMetadata>	\[\]	Extracted hyperlinks with type classification
images	Array<ImageMetadataType>	\[\]	Extracted images with source and dimensions
structuredData	Array<StructuredData>	\[\]	Extracted structured data blocks

---

HtmlOutputConfig

Configuration for styled HTML output.

When set on html_output alongside output_format = OutputFormat.Html, the pipeline builds a StyledHtmlRenderer instead of the plain comrak-based renderer.

Field	Type	Default	Description
css	string | null	null	Inline CSS string injected into the output after the theme stylesheet. Concatenated after css_file content when both are set.
cssFile	string | null	null	Path to a CSS file loaded once at renderer construction time. Concatenated before css when both are set.
theme	HtmlTheme	HtmlTheme.Unstyled	Built-in colour/typography theme. Default: HtmlTheme.Unstyled.
classPrefix	string	—	CSS class prefix applied to every emitted class name. Default: "kb-". Change this if your host application already uses classes that start with kb-.
embedCss	boolean	true	When true (default), write the resolved CSS into a <style> block immediately after the opening <div class="{prefix}doc">. Set to false to emit only the structural markup and wire up your own stylesheet targeting the kb-* class names.

Methods

default()

Signature:

static default(): HtmlOutputConfig

Example:

const result = HtmlOutputConfig.default();

Returns: HtmlOutputConfig

---

ImageExtractionConfig

Image extraction configuration.

Field	Type	Default	Description
extractImages	boolean	true	Extract images from documents
targetDpi	number	300	Target DPI for image normalization
maxImageDimension	number	4096	Maximum dimension for images (width or height)
injectPlaceholders	boolean	true	Whether to inject image reference placeholders into markdown output. When true (default), image references like !\[Image 1\](embedded:p1_i0) are appended to the markdown. Set to false to extract images as data without polluting the markdown output.
autoAdjustDpi	boolean	true	Automatically adjust DPI based on image content
minDpi	number	72	Minimum DPI threshold
maxDpi	number	600	Maximum DPI threshold
maxImagesPerPage	number | null	null	Maximum number of image objects to extract per PDF page. Some PDFs (e.g. technical diagrams stored as thousands of raster fragments) can trigger extremely long or indefinite extraction times when every image object on a dense page is decoded individually via the PDF extractor. Setting this limit causes xberg to stop collecting individual images once the count per page reaches the cap and emit a warning instead. null (default) means no limit — all images are extracted.
classify	boolean	false	When true, extracted images are classified by kind and grouped into clusters where they appear to belong to one figure. Defaults to false — opt in explicitly to avoid unexpected ML overhead.
includePageRasters	boolean	false	When true, full-page renders produced during OCR preprocessing are captured and returned as ImageKind.PageRaster entries in ExtractedDocument.images. PDF + OCR only. No rasters are captured for non-PDF inputs or when the document-level OCR bypass is active (whole-document backend). When OCR is enabled and this flag is set but the active backend skips per-page rendering, a ProcessingWarning is emitted in ExtractedDocument.processing_warnings. Defaults to false. Enable when downstream consumers need page thumbnails (e.g. citation previews, visual grounding).
runOcrOnImages	boolean	true	Run OCR on extracted images and include the recognized text in the document content. When true (default) and ExtractionConfig.ocr is configured, extracted images are processed with the configured OCR backend. Set to false to extract images without OCR processing, even when OCR is enabled.
ocrTextOnly	boolean	false	When true, image OCR results are rendered as plain text without the !\[...\](...) markdown placeholder. Only takes effect when run_ocr_on_images is also true.
appendOcrText	boolean	false	When true and ocr_text_only is false, append the OCR text after the image placeholder in the rendered output.
outputFormat	ImageOutputFormat	ImageOutputFormat.Native	Target format for re-encoding extracted images. When set to anything other than Native, each extracted image is re-encoded to the requested format before being returned. This lets callers receive uniform output without duplicating encode logic downstream. Defaults to Native — no re-encode pass is performed and ExtractedImage.format reflects the source extractor’s output.
svg	SvgOptions	—	SVG-specific knobs for the image-encode pipeline. Controls sanitization and rasterization DPI when the source or output format is SVG. Only available when the svg feature is active.
includeDataBase64	boolean	false	When true, populate ExtractedImage.data_base64 with a Base64-encoded copy of the raw image bytes. Useful for JSON-only clients that cannot efficiently parse the default integer-array serialization of data. Defaults to false; enabling it doubles the in-memory image representation for the duration of the response.

Methods

default()

Signature:

static default(): ImageExtractionConfig

Example:

const result = ImageExtractionConfig.default();

Returns: ImageExtractionConfig

---

ImageMetadata

Image metadata extracted from image files.

Includes dimensions, format, and EXIF data.

Field	Type	Default	Description
width	number	—	Image width in pixels
height	number	—	Image height in pixels
format	string	—	Image format (e.g., “PNG”, “JPEG”, “TIFF”)
exif	Record<string, string>	{}	EXIF metadata tags

---

ImageMetadataType

Image element metadata.

Field	Type	Default	Description
src	string	—	Image source (URL, data URI, or SVG content)
alt	string | null	null	Alternative text from alt attribute
title	string | null	null	Title attribute
imageType	ImageType	—	Image type classification

---

ImagePreprocessingConfig

Image preprocessing configuration for OCR.

These settings control how images are preprocessed before OCR to improve text recognition quality. Different preprocessing strategies work better for different document types.

Field	Type	Default	Description
targetDpi	number	300	Target DPI for the image (300 is standard, 600 for small text).
autoRotate	boolean	false	Auto-detect and correct image rotation.
deskew	boolean	true	Correct skew (tilted images).
denoise	boolean	false	Remove noise from the image.
contrastEnhance	boolean	false	Enhance contrast for better text visibility.
binarizationMethod	string	"otsu"	Binarization method: “otsu”, “sauvola”, “adaptive”.
invertColors	boolean	false	Invert colors (white text on black → black on white).

Methods

default()

Signature:

static default(): ImagePreprocessingConfig

Example:

const result = ImagePreprocessingConfig.default();

Returns: ImagePreprocessingConfig

---

ImagePreprocessingMetadata

Image preprocessing metadata.

Tracks the transformations applied to an image during OCR preprocessing, including DPI normalization, resizing, and resampling.

Field	Type	Default	Description
targetDpi	number	—	Target DPI from configuration
scaleFactor	number	—	Scaling factor applied to the image
autoAdjusted	boolean	—	Whether DPI was auto-adjusted based on content
finalDpi	number	—	Final DPI after processing
resampleMethod	string	—	Resampling algorithm used (“LANCZOS3”, “CATMULLROM”, etc.)
dimensionClamped	boolean	—	Whether dimensions were clamped to max_image_dimension
calculatedDpi	number | null	null	Calculated optimal DPI (if auto_adjust_dpi enabled)
skippedResize	boolean	—	Whether resize was skipped (dimensions already optimal)
resizeError	string | null	null	Error message if resize failed

---

InlineElement

Inline element within a block.

Represents text with formatting, links, images, etc.

Field	Type	Default	Description
elementType	InlineType	—	Type of inline element
content	string	—	Text content
metadata	Record<string, string> | null	null	Additional metadata (e.g., href for links, src/alt for images)

---

JatsMetadata

JATS (Journal Article Tag Suite) metadata.

Field	Type	Default	Description
copyright	string | null	null	Copyright statement from the article’s <permissions> element.
license	string | null	null	Open-access license URI from the article’s <license> element.
historyDates	Record<string, string>	{}	Publication history dates keyed by event type (e.g. "received", "accepted").
contributorRoles	Array<ContributorRole>	\[\]	Authors and contributors with their stated roles.

---

Keyword

Extracted keyword with metadata.

Field	Type	Default	Description
text	string	—	The keyword text.
score	number	—	Relevance score (higher is better, algorithm-specific range).
algorithm	KeywordAlgorithm	—	Algorithm that extracted this keyword.
positions	Array<number> | null	null	Optional positions where keyword appears in text (character offsets).

---

KeywordConfig

Keyword extraction configuration.

Field	Type	Default	Description
algorithm	KeywordAlgorithm	KeywordAlgorithm.Yake	Algorithm to use for extraction.
maxKeywords	number	10	Maximum number of keywords to extract (default: 10).
minScore	number	0	Minimum score threshold (0.0-1.0, default: 0.0). Keywords with scores below this threshold are filtered out. Note: Score ranges differ between algorithms.
language	string | null	"en"	Language code for stopword filtering (e.g., “en”, “de”, “fr”). If None, no stopword filtering is applied.
yakeParams	YakeParams | null	null	YAKE-specific tuning parameters.
rakeParams	RakeParams | null	null	RAKE-specific tuning parameters.

Methods

default()

Signature:

static default(): KeywordConfig

Example:

const result = KeywordConfig.default();

Returns: KeywordConfig

---

LanguageConfidence

Since: v1.1

Structured per-language detection result: confidence, document share, and script — the information the ISO-code-only detected_languages list cannot convey (#261).

Populated by language_detection alongside detected_languages, with one entry per language, in the same order as detected_languages.

Field	Type	Default	Description
language	string	—	ISO 639-3 language code, matching the corresponding entry in detected_languages.
confidence	number	—	Confidence for this language, in \[0.0, 1.0\]. In single-language mode this is whatlang’s Info.confidence() for the whole document. In multi-language mode this is the average whatlang confidence across the document’s 200-character chunks that were classified as this language.
proportion	number	—	Share of the document’s analyzed content classified as this language, in \[0.0, 1.0\]. In single-language mode this is always 1.0. In multi-language mode this is the fraction of 200-character chunks classified as this language (chunks that did not meet min_confidence for any language are excluded from the count but still count toward the denominator).
script	string	—	Writing system whatlang detected for this language (e.g. "Latin", "Cyrillic").
reliable	boolean	—	Whether this detection is considered reliable. In single-language mode this is whatlang’s own Info.is_reliable() (confidence above whatlang’s internal 0.9 threshold). In multi-language mode this is the chunk-averaged confidence above that same 0.9 threshold, since whatlang’s is_reliable() only applies to a single detection.

---

LanguageDetectionConfig

Language detection configuration.

Field	Type	Default	Description
enabled	boolean	true	Enable language detection
minConfidence	number	0.8	Minimum confidence threshold (0.0-1.0)
detectMultiple	boolean	false	Detect multiple languages in the document

Methods

default()

Signature:

static default(): LanguageDetectionConfig

Example:

const result = LanguageDetectionConfig.default();

Returns: LanguageDetectionConfig

---

LateInteractionConfig

Configuration for the late-interaction (ColBERT) pipeline.

Controls which model to use, batching, and download/cache behavior for the local ONNX ColBERT model.

Since v5.0.

Field	Type	Default	Description
model	LateInteractionModelType	LateInteractionModelType.Preset	The late-interaction model to use (defaults to the “gte-moderncolbert” preset).
batchSize	number	—	Batch size for local ONNX inference. ColBERT emits a \[seq, dim\] multi-vector embedding per document, so memory scales with batch size — keep this modest.
maxLength	number	—	Maximum token sequence length for the tokenizer (documents).
queryMaxLength	number	—	Fixed padded length for query augmentation. ColBERT queries are padded (with the mask token, kept attention-live) to exactly this many tokens rather than truncated/left as-is — this is the “query augmentation” trick from the ColBERT paper.
showDownloadProgress	boolean	false	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at info level on the xberg.model_download target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by LateInteractionModelType.Plugin, which downloads no model.
cacheDir	string | null	null	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
acceleration	AccelerationConfig | null	null	Hardware acceleration for the late-interaction ONNX model.
maxEmbedDurationSecs	number | null	null	Maximum wall-clock duration (in seconds) for a single embed call when using LateInteractionModelType.Plugin. null disables the timeout.

Methods

default()

Signature:

static default(): LateInteractionConfig

Example:

const result = LateInteractionConfig.default();

Returns: LateInteractionConfig

---

LateInteractionMatch

A single document match returned by max_sim_rank, with its position in the input and MaxSim score.

Since v5.0.

Field	Type	Default	Description
index	number	—	Position of this document in the original input slice.
score	number	—	MaxSim relevance score. Higher means more relevant to the query.

---

LateInteractionPreset

Static metadata for a bundled ColBERT preset (WASM/Android-safe, no ORT).

Since v5.0.

Field	Type	Default	Description
name	string	—	Stable preset name referenced from config.
modelRepo	string	—	HuggingFace repository hosting the ONNX model.
modelFile	string	—	Path to the ONNX file within the repo.
additionalFiles	Array<string>	—	Sibling files that must be downloaded alongside model_file.
maxLength	number	—	Maximum document token sequence length.
queryMaxLength	number	—	Fixed padded query length (ColBERT query augmentation).
dim	number	—	Per-token embedding dimensionality.
description	string	—	Human-readable description.

---

LayoutDetection

A single layout detection result.

Field	Type	Default	Description
className	LayoutClass	—	Detected layout class (e.g. Table, Text, Title).
confidence	number	—	Detection confidence score in \[0.0, 1.0\].
bbox	BBox	—	Bounding box in image pixel coordinates.

---

LayoutDetectionConfig

Layout detection configuration.

Controls layout detection behavior in the extraction pipeline. When set on ExtractionConfig, layout detection is enabled for PDF extraction.

Field	Type	Default	Description
strategy	LayoutStrategy	LayoutStrategy.Always	Which pages the layout model runs on. Defaults to LayoutStrategy.Always, the historical behavior: every page is rendered and inferred. LayoutStrategy.Auto pre-screens pages with cheap signals and skips the model where it cannot help.
confidenceThreshold	number | null	null	Confidence threshold override (None = use model default).
applyHeuristics	boolean	true	Whether to apply postprocessing heuristics (default: true).
tableModel	TableModel	TableModel.Tatr	Table structure recognition model. Controls which model is used for table cell detection within layout-detected table regions. Defaults to TableModel.Tatr.
tableOverlapPreference	TableOverlapPreference	TableOverlapPreference.Content	How to resolve overlapping native vs layout tables. When a native oxide table and a layout (TATR/SLANeXT) table overlap on the same region, this controls which one is kept. Defaults to TableOverlapPreference.Content (historical behavior: keep the table with more content). Set to TableOverlapPreference.Native to favor source reading order (higher text F1) over the model’s cell reflow.
acceleration	AccelerationConfig | null	null	Hardware acceleration for ONNX models (layout detection + table structure). When set, controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for inference. Defaults to null (auto-select per platform).
enableChartUnderstanding	boolean	false	Route regions classified as charts to the chart-understanding OCR task. When true, layout regions detected as charts are sent to the VLM chart task (data-series/axis recovery) instead of being treated as generic image regions. Defaults to false — chart understanding is opt-in and has no effect on standard text/table extraction scores.

Methods

default()

Signature:

static default(): LayoutDetectionConfig

Example:

const result = LayoutDetectionConfig.default();

Returns: LayoutDetectionConfig

---

LayoutRegion

A detected layout region on a page.

When layout detection is enabled, each page may have layout regions identifying different content types (text, pictures, tables, etc.) with confidence scores and spatial positions.

Field	Type	Default	Description
className	string	—	Layout class name (e.g. “picture”, “table”, “text”, “section_header”).
confidence	number	—	Confidence score from the layout detection model (0.0 to 1.0).
boundingBox	BoundingBox	—	Bounding box in document coordinate space.
areaFraction	number	—	Fraction of the page area covered by this region (0.0 to 1.0).

---

LinkMetadata

Link element metadata.

Field	Type	Default	Description
href	string	—	The href URL value
text	string	—	Link text content (normalized)
title	string | null	null	Optional title attribute
linkType	LinkType	—	Link type classification
rel	Array<string>	—	Rel attribute values

---

LlmBudgetConfig

Since: v1.1

Budget enforcement configuration.

Mirrors liter-llm’s LlmBudgetConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not enforced at request time.

Field	Type	Default	Description
globalLimit	number | null	null	Global spend limit in USD.
modelLimits	Record<string, number> | null	{}	Per-model spend limits in USD, keyed by model name.
enforcement	string | null	null	Enforcement mode: "hard" (reject over-budget requests) or "soft" (log only).

---

LlmCacheConfig

Since: v1.1

Response cache configuration.

Mirrors liter-llm’s LlmCacheConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not consulted at request time.

Field	Type	Default	Description
maxEntries	number | null	null	Maximum number of cached entries.
ttlSeconds	number | null	null	Cache entry time-to-live, in seconds.
backend	string | null	null	Cache backend name (e.g. "memory", or an opendal scheme).
backendConfig	Record<string, string> | null	{}	Backend-specific configuration key/value pairs.

---

LlmConfig

Configuration for an LLM provider/model via liter-llm.

Each feature (VLM OCR, VLM embeddings, structured extraction) carries its own LlmConfig, allowing different providers per feature.

Debug is implemented by hand so api_key, header values, and the AWS credentials in BedrockConfig are never printed.

Field	Type	Default	Description
model	string	—	Provider/model string using liter-llm routing format. Examples: "openai/gpt-4o", "anthropic/claude-sonnet-4-20250514", "groq/llama-3.1-70b-versatile".
apiKey	string | null	null	API key for the provider. When null, liter-llm falls back to the provider’s standard environment variable (e.g., OPENAI_API_KEY).
baseUrl	string | null	null	Custom base URL override for the provider endpoint.
timeoutSecs	number | null	null	Request timeout in seconds. When null, liter-llm’s built-in 60s default applies, except the VLM OCR path which uses a 300s default (a single page image transcription routinely exceeds 60s). Set explicitly to override.
maxRetries	number | null	null	Maximum retry attempts (default: 3).
temperature	number | null	null	Sampling temperature for generation tasks.
maxTokens	number | null	null	Maximum tokens to generate.
topP	number | null	null	Nucleus sampling parameter for generation tasks, applied to individual requests built from this config. Restricts sampling to the smallest set of tokens whose cumulative probability mass is at least this value; lower is more focused. Validated to \[0.0, 1.0\] by LlmConfig.validate. Mirrors liter-llm’s ChatCompletionRequest.top_p. A request-time parameter like temperature/max_tokens above, not a client-level setting.
stop	Array<string> | null	\[\]	Stop sequence(s) that halt token generation, applied to individual requests built from this config. Mirrors liter-llm’s ChatCompletionRequest.stop (types.common.StopSequence), which liter-llm represents as either a single string or a list of strings via an untagged enum. Always expressed here as a list — even one stop sequence is \["..."\] — so the field has a single, FFI-friendly shape across every language binding instead of a single-or-list union type. Converted to liter-llm’s StopSequence.Multiple at each request-building call site; see to_stop_sequence.
seed	number | null	null	Random seed for reproducible outputs, applied to individual requests built from this config. Provider support varies — some silently ignore it. Mirrors liter-llm’s ChatCompletionRequest.seed.
presencePenalty	number | null	null	Presence penalty for generation tasks, applied to individual requests built from this config. Positive values discourage the model from repeating topics already present in the conversation. Validated to \[-2.0, 2.0\] by LlmConfig.validate. Mirrors liter-llm’s ChatCompletionRequest.presence_penalty.
frequencyPenalty	number | null	null	Frequency penalty for generation tasks, applied to individual requests built from this config. Positive values discourage the model from repeating the same tokens verbatim. Validated to \[-2.0, 2.0\] by LlmConfig.validate. Mirrors liter-llm’s ChatCompletionRequest.frequency_penalty.
reasoningEffort	string | null	null	Reasoning effort level for extended-thinking models, applied to individual requests built from this config. Mirrors liter-llm’s ChatCompletionRequest.reasoning_effort (types.chat.ReasoningEffort). A request-time parameter like temperature/ max_tokens above, not a client-level setting — into_client_builder does not map it. Accepted as a plain string — one of "low", "medium", "high", "minimal", "max" (case-insensitive; liter-llm’s own #\[serde(rename_all = "lowercase")\] spelling) — rather than importing liter-llm’s enum, because this module compiles even when the liter-llm feature is disabled. See parse_reasoning_effort for the conversion into liter_llm.ReasoningEffort.
extraBody	unknown | null	null	Provider-specific extra parameters merged into the request body (guardrails, safety settings, grounding config, etc.), applied to individual requests built from this config. Mirrors liter-llm’s ChatCompletionRequest.extra_body. A request-time parameter like temperature/max_tokens above, not a client-level setting.
loadEnv	boolean | null	null	Whether liter-llm should load provider credentials from environment variables. Mirrors liter-llm’s ClientConfigBuilder.load_env. When null, liter-llm’s own default behavior applies.
headers	Record<string, string> | null	{}	Extra HTTP headers sent with every request to the provider. Mirrors liter-llm’s ClientConfigBuilder.header, for gateways or providers that require custom auth/routing headers.
providers	Array<LlmProviderConfig> | null	\[\]	Custom provider configurations, in addition to liter-llm’s built-in providers. Mirrors liter-llm’s LlmConfig.providers, for OpenAI-compatible gateways and self-hosted model servers that are not in the built-in provider catalog.
cache	LlmCacheConfig | null	null	Response cache configuration. Mirrors liter-llm’s LlmConfig.cache. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused. Boxed for the same reason as bedrock (a4579589ac): LlmConfig is the payload of EmbeddingModelType.Llm and RerankerModelType.Llm, whose other variants are tens of bytes. Inlining this and the two sub-configs below pushed that variant to 480 bytes and tripped clippy.large_enum_variant on the --features full leg. ~keep
budget	LlmBudgetConfig | null	null	Budget enforcement configuration. Mirrors liter-llm’s LlmConfig.budget. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
rateLimit	LlmRateLimitConfig | null	null	Per-model rate limiting configuration. Mirrors liter-llm’s LlmConfig.rate_limit. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
costTracking	boolean | null	null	Enable per-request cost tracking. Mirrors liter-llm’s LlmConfig.cost_tracking. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
tracing	boolean | null	null	Enable OpenTelemetry-compatible tracing spans. Mirrors liter-llm’s LlmConfig.tracing. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
cooldownSecs	number | null	null	Cooldown duration after transient errors, in seconds. Mirrors liter-llm’s LlmConfig.cooldown_secs. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
healthCheckSecs	number | null	null	Background health check interval, in seconds. Mirrors liter-llm’s LlmConfig.health_check_secs. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value is accepted but unused.
bedrock	BedrockConfig | null	null	AWS Bedrock settings (region, cross-region routing, explicit credentials). Only consulted for bedrock/-prefixed models. When null — or when an individual field inside it is null — liter-llm falls back to the standard AWS environment variables and the default credential chain.
credentialProvider	CredentialProviderConfig | null	null	Managed OAuth2/STS credential provider for auth modes liter-llm cannot express via a static api_key — Azure AD, Vertex AI OAuth2, Vertex AI Application Default Credentials, and AWS STS Web Identity (EKS IRSA) for Bedrock. Mirrors liter-llm’s client.ClientConfigBuilder.credential_provider, which takes an Arc<dyn liter_llm.auth.CredentialProvider> trait object — that cannot appear in a serde DTO. Every CredentialProviderConfig variant is plain data instead, so it round-trips through TOML/JSON/YAML and every language binding like the rest of LlmConfig. Inert on wasm32: crate.llm (the module that reads this field — build_credential_provider and friends) is compiled out entirely on that target, via the crate-root #\[cfg(all(feature = "liter-llm", not(target_arch = "wasm32")))\] pub mod llm; gate in lib.rs. Every variant needs liter-llm’s native-http-backed auth modules, and wasm32 builds request only wasm-http (see the liter-llm dependency comment in Cargo.toml), so there is no code path left on that target to construct a provider from this field, or to reject it. This type (core.config.llm) has no liter-llm dependency itself and compiles on every target, so setting this field on a wasm32 build is accepted by serde and silently ignored — a plain no-op, not a Validation. Reject a wasm32 build that sets this field yourself if that silence is a problem for your use case; xberg does not do it for you. GitHub Copilot’s device-flow provider has no variant here: it takes no configuration at all (liter_llm.auth.github_copilot.GithubCopilotCredentialProvider.new accepts only an HTTP client) and drives an interactive terminal prompt, so it cannot be expressed as data. A Rust embedder who needs it — or any other fully custom CredentialProvider — can call xberg.llm.client.create_client_with_credential_provider directly with a liter-llm dependency of their own.

Methods

validate()

Validate the request-time sampling parameters that have a documented range: top_p ([0.0, 1.0]), presence_penalty, and frequency_penalty (both [-2.0, 2.0], matching liter-llm’s/OpenAI’s semantics). An unset field is always valid — silence in config should never be rejected.

Called from build_client_config before a liter-llm client is built from this config, alongside the existing validate_cache_backend check in that function.

Signature:

validate(): void

Example:

instance.validate();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

LlmProviderConfig

Since: v1.1

A custom provider configuration entry, in addition to liter-llm’s built-in providers.

Mirrors liter-llm’s LlmProviderConfig.

Field	Type	Default	Description
name	string	—	Provider name, used to key model prefix matching.
baseUrl	string	—	Base URL for the provider’s OpenAI-compatible API.
authHeader	string | null	null	Header name used to carry the API key (defaults to Authorization when unset).
modelPrefixes	Array<string>	\[\]	Model name prefixes routed to this provider (e.g. \["my-provider/"\]).

---

LlmRateLimitConfig

Since: v1.1

Per-model rate limiting configuration.

Mirrors liter-llm’s LlmRateLimitConfig. Only takes effect when liter-llm’s tower feature is compiled in; otherwise the value round-trips through configuration but is not enforced at request time.

Field	Type	Default	Description
rpm	number | null	null	Requests per minute limit.
tpm	number | null	null	Tokens per minute limit.
windowSeconds	number | null	null	Rate limit window, in seconds.

---

LlmUsage

Token usage and cost data for a single LLM call made during extraction.

Populated when VLM OCR, structured extraction, or LLM-based embeddings are used. Multiple entries may be present when multiple LLM calls occur within one extraction (e.g. VLM OCR + structured extraction).

Field	Type	Default	Description
model	string	—	The LLM model identifier (e.g. “openai/gpt-4o”, “anthropic/claude-sonnet-4-20250514”).
source	string	—	The pipeline stage that triggered this LLM call (e.g. “vlm_ocr”, “structured_extraction”, “embeddings”).
inputTokens	number | null	null	Number of input/prompt tokens consumed.
outputTokens	number | null	null	Number of output/completion tokens generated.
totalTokens	number | null	null	Total tokens (input + output).
estimatedCost	number | null	null	Estimated cost in USD based on the provider’s published pricing.
finishReason	string | null	null	Why the model stopped generating (e.g. “stop”, “length”, “content_filter”).

---

MapResult

The result of a map operation, containing discovered URLs.

Field	Type	Default	Description
urls	Array<SitemapUrl>	\[\]	The list of discovered URLs.

---

MetaSchema

Compiled meta-schema validator over preset.schema.json.

Methods

compile()

Compile the given JSON text as a Draft 2020-12 meta-schema.

Signature:

static compile(metaSchemaJson: string): MetaSchema

Example:

const result = MetaSchema.compile("value");

Parameters:

Name	Type	Required	Description
metaSchemaJson	string	Yes	The meta schema json

Returns: MetaSchema

Errors: Throws Error with a descriptive message.

parsePreset()

Validate raw against the meta-schema and deserialize into a Preset, stamping the fingerprint over the canonical file bytes.

Signature:

parsePreset(path: string, raw: Buffer): Preset

Example:

const result = instance.parsePreset("value", Buffer.from("data"));

Parameters:

Name	Type	Required	Description
path	string	Yes	Path to the file
raw	Buffer	Yes	The raw

Returns: Preset

Errors: Throws Error with a descriptive message.

---

Metadata

Extraction result metadata.

Contains common fields applicable to all formats, format-specific metadata via a discriminated union, and additional custom fields from postprocessors.

Field	Type	Default	Description
title	string | null	null	Document title
subject	string | null	null	Document subject or description
authors	Array<string> | null	\[\]	Primary author(s) - always Vec for consistency
keywords	Array<string> | null	\[\]	Keywords/tags - always Vec for consistency
language	string | null	null	Primary language (ISO 639 code)
createdAt	string | null	null	Creation timestamp (ISO 8601 format)
modifiedAt	string | null	null	Last modification timestamp (ISO 8601 format)
createdBy	string | null	null	User who created the document
modifiedBy	string | null	null	User who last modified the document
pages	PageStructure | null	null	Page/slide/sheet structure with boundaries
format	FormatMetadata | null	null	Format-specific metadata (discriminated union) Contains detailed metadata specific to the document format. Serialized as a nested "format" object with a format_type discriminator field.
imagePreprocessing	ImagePreprocessingMetadata | null	null	Image preprocessing metadata (when OCR preprocessing was applied)
jsonSchema	unknown | null	null	JSON schema (for structured data extraction)
error	ErrorMetadata | null	null	Error metadata (for batch operations)
extractionDurationMs	number | null	null	Extraction duration in milliseconds (for benchmarking). This field is populated by batch extraction to provide per-file timing information. It’s null for single-file extraction (which uses external timing).
category	string | null	null	Document category (from frontmatter or classification).
tags	Array<string> | null	\[\]	Document tags (from frontmatter).
documentVersion	string | null	null	Document version string (from frontmatter).
abstractText	string | null	null	Abstract or summary text (from frontmatter).
outputFormat	string | null	null	Output format identifier (e.g., “markdown”, “html”, “text”). Set by the output format pipeline stage when format conversion is applied. Previously stored in metadata.additional\["output_format"\].
ocrUsed	boolean	—	Whether OCR was used during extraction. Set to true whenever the extraction pipeline ran an OCR backend (Tesseract, PaddleOCR, VLM, etc.) and used that output as the primary or fallback text. false means native text extraction was used exclusively.
additional	Record<string, unknown>	{}	Additional custom fields from postprocessors. Serialized as a nested "additional" object (not flattened at root level). Uses Cow<'static, str> keys so static string keys avoid allocation.

Methods

isEmpty()

Returns true when no metadata fields, format-specific metadata, or additional postprocessor fields are populated.

Signature:

isEmpty(): boolean

Example:

const result = instance.isEmpty();

Returns: boolean

---

ModelPaths

Combined paths to all models needed for OCR (backward compatibility).

Field	Type	Default	Description
detModel	string	—	Exact path to the detection ONNX model in the Hugging Face snapshot.
clsModel	string	—	Exact path to the classification ONNX model in the Hugging Face snapshot.
recModel	string	—	Exact path to the recognition ONNX model in the Hugging Face snapshot.
dictFile	string	—	Path to the character dictionary file.

---

MultiVectorEmbedding

A ColBERT multi-vector embedding: one row per attention-live token.

data is a flat, row-major buffer of length num_tokens * dim — row i (the embedding for token i) occupies data[i*dim .. (i+1)*dim]. Flat storage keeps the type FFI-friendly across binding boundaries; use MultiVectorEmbedding.rows internally to iterate per-token slices.

Since v5.0.

Field	Type	Default	Description
numTokens	number	—	Number of attention-live token rows (padding rows are dropped, not zeroed — see engine.normalize_tokens).
dim	number	—	Dimensionality of each per-token vector.
data	Array<number>	—	Flat row-major buffer, length num_tokens * dim.

Methods

isWellFormed()

Returns true if data holds exactly num_tokens * dim values — i.e. the flat buffer matches the declared shape.

All fields are pub and the type is Deserialize, so a value coming from an untrusted source (FFI caller, JSON, a store row) may be malformed. max_sim_score guards on this so a length-mismatched buffer scores 0.0 rather than silently mis-scoring (a shorter data would make rows’ chunks_exact drop a trailing partial chunk). Uses checked_mul so an overflowing num_tokens * dim is reported as malformed instead of wrapping.

Since v5.0.

Signature:

isWellFormed(): boolean

Example:

const result = instance.isWellFormed();

Returns: boolean

---

MultidocInput

Input signals for multi-document boundary detection.

Field	Type	Default	Description
pageCount	number	—	Total number of pages in the PDF.
pages	Array<PageSignals>	—	Per-page signals extracted from the PDF.

---

MultidocThresholds

Thresholds for multi-document boundary detection.

All fields are public; callers override any subset via struct-update syntax.

Field	Type	Default	Description
densityShiftThreshold	number	0.3	Text density difference threshold for DensityShift detection. Default: 0.3.
bigramOverlapMin	number	0.1	Minimum bigram-overlap ratio below which a density shift is promoted to a DensityShift boundary. Default: 0.1 (10 % overlap).

Methods

default()

Signature:

static default(): MultidocThresholds

Example:

const result = MultidocThresholds.default();

Returns: MultidocThresholds

---

NerConfig

Since: v1.0

Configuration for the NER post-processor.

Field	Type	Default	Description
backend	NerBackendKind	NerBackendKind.Onnx	Backend that runs the entity detection.
categories	Array<EntityCategory>	\[\]	Entity categories to detect. Defaults to a sensible PERSON/ORG/LOCATION/EMAIL set when empty.
model	string | null	null	Override the default model — only used by NerBackendKind.Onnx. null lets the backend pick its pinned default xberg GLiNER model alias.
llm	LlmConfig | null	null	Optional LLM configuration — only used by NerBackendKind.Llm. Token usage for LLM backends is recorded in ExtractedDocument.llm_usage.
customLabels	Array<string>	\[\]	Arbitrary user-supplied entity labels for zero-shot detection. xberg-gliner natively supports zero-shot inference over caller-supplied labels. The LLM backend also honours these labels by including them in the structured-output schema. Custom labels surface as EntityCategory.Custom in the resulting Entity stream. Use this when you need domain-specific entity types (e.g. "Treatment", "Product", "Vessel") without forking GLiNER’s taxonomy.

---

OcrBackend

Trait for OCR backend plugins.

Implement this trait to add custom OCR capabilities. OCR backends can be:

• Native Rust implementations (like Tesseract)
• FFI bridges to external libraries (like PaddleOCR)
• Cloud-based OCR services (Google Vision, AWS Textract, etc.)

Thread Safety

OCR backends must be thread-safe (Send + Sync) to support concurrent processing.

Methods

processImage()

Process an image and extract text via OCR.

Returns:

An ExtractedDocument containing the extracted text and metadata.

Errors:

• XbergError.Ocr - OCR processing failed
• XbergError.Validation - Invalid image format or configuration
• XbergError.Io - I/O errors (these always bubble up)

Reading backend_options

Backends that support runtime tuning can read config.backend_options and deserialize only the keys they care about. Unknown keys are silently ignored, so multiple backends can coexist in a pipeline without key conflicts.

Signature:

processImage(imageBytes: Buffer, config: OcrConfig): Promise<ExtractedDocument>

Example:

const result = await instance.processImage(Buffer.from("data"), new OcrConfig());

Parameters:

Name	Type	Required	Description
imageBytes	Buffer	Yes	Raw image data (JPEG, PNG, TIFF, etc.)
config	OcrConfig	Yes	OCR configuration (language, PSM mode, etc.)

Returns: ExtractedDocument

Errors: Throws Error with a descriptive message.

processImageFile()

Process a file and extract text via OCR.

Default implementation reads the file and calls process_image. Override for custom file handling or optimizations.

Errors:

Same as process_image, plus file I/O errors.

Signature:

processImageFile(path: string, config: OcrConfig): Promise<ExtractedDocument>

Example:

const result = await instance.processImageFile("value", new OcrConfig());

Parameters:

Name	Type	Required	Description
path	string	Yes	Path to the image file
config	OcrConfig	Yes	OCR configuration

Returns: ExtractedDocument

Errors: Throws Error with a descriptive message.

supportsLanguage()

Check if this backend supports a given language code.

Returns:

true if the language is supported, false otherwise.

Signature:

supportsLanguage(lang: string): boolean

Example:

const result = instance.supportsLanguage("value");

Parameters:

Name	Type	Required	Description
lang	string	Yes	ISO 639-2/3 language code (e.g., “eng”, “deu”, “fra”)

Returns: boolean

backendType()

Get the backend type identifier.

Returns:

The backend type enum value.

Signature:

backendType(): OcrBackendType

Example:

const result = instance.backendType();

Returns: OcrBackendType

supportedLanguages()

Optional: Get a list of all supported languages.

Defaults to empty list. Override to provide comprehensive language support info.

Signature:

supportedLanguages(): Array<string>

Example:

const result = instance.supportedLanguages();

Returns: Array<string>

supportsTableDetection()

Optional: Check if the backend supports table detection.

Defaults to false. Override if your backend can detect and extract tables.

Signature:

supportsTableDetection(): boolean

Example:

const result = instance.supportsTableDetection();

Returns: boolean

supportsDocumentProcessing()

Check if the backend supports direct document-level processing (e.g. for PDFs).

Defaults to false. Override if the backend has optimized document processing.

Signature:

supportsDocumentProcessing(): boolean

Example:

const result = instance.supportsDocumentProcessing();

Returns: boolean

emitsStructuredMarkdown()

Declare that this backend emits structured markdown directly (tables, headings, lists) and downstream layout reconstruction should be skipped.

Defaults to false — classical OCR backends (Tesseract, PaddleOCR classical) return plain text per detected region. End-to-end VLM backends (PaddleOCR-VL, GOT-OCR 2.0) emit markdown in one forward pass and should override this to true.

Signature:

emitsStructuredMarkdown(): boolean

Example:

const result = instance.emitsStructuredMarkdown();

Returns: boolean

processDocument()

Process a document file directly via OCR.

Only called if supports_document_processing returns true.

Signature:

processDocument(path: string, config: OcrConfig): Promise<ExtractedDocument>

Example:

const result = await instance.processDocument("value", new OcrConfig());

Parameters:

Name	Type	Required	Description
path	string	Yes	The path
config	OcrConfig	Yes	The ocr config

Returns: ExtractedDocument

Errors: Throws Error with a descriptive message.

---

OcrConfidence

Confidence scores for an OCR element.

Separates detection confidence (how confident that text exists at this location) from recognition confidence (how confident about the actual text content).

Field	Type	Default	Description
detection	number | null	null	Detection confidence: how confident the OCR engine is that text exists here. PaddleOCR provides this as box_score, Tesseract doesn’t have a direct equivalent. Range: 0.0 to 1.0 (or None if not available).
recognition	number	—	Recognition confidence: how confident about the text content. Range: 0.0 to 1.0.

---

OcrConfig

OCR configuration.

Field	Type	Default	Description
enabled	boolean	true	Whether OCR is enabled. Setting enabled: false is a shorthand for disable_ocr: true on the parent ExtractionConfig. Images return metadata only; PDFs use native text extraction without OCR fallback. Defaults to true. When false, all other OCR settings are ignored.
backend	string	—	OCR backend: tesseract, paddleocr, paddle-ocr, sceptre, or vlm. Sceptre uses ONNX Runtime on desktop/server and tract on supported mobile builds. Browser WebAssembly uses the separate byte-fed Sceptre worker API.
language	Array<string>	\[\]	Language code(s) for OCR recognition. Defaults to \["eng"\]. For Tesseract, languages are joined with “+”. A list is the canonical form and the only form accepted by the binding object APIs (Python, Node, PHP, WASM, etc.): \["eng", "deu"\]. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”).
tesseractConfig	TesseractConfig | null	null	Tesseract-specific configuration (optional)
outputFormat	OutputFormat | null	null	Output format for OCR results (optional, for format conversion)
paddleOcrConfig	unknown | null	null	PaddleOCR-specific configuration (optional, JSON passthrough). Deserialized into a PaddleOcrConfig, so any of its fields can be overridden here — most notably model_version ("pp-ocrv6" default / "pp-ocrv5") and model_tier. In TOML: toml \[ocr.paddle_ocr_config\] model_version = "pp-ocrv5" model_tier = "server" The XBERG_OCR_MODEL_VERSION / XBERG_OCR_MODEL_TIER environment variables set the same two keys for env-configured servers (issue #1279).
backendOptions	unknown | null	null	Arbitrary per-call options passed through to the backend unchanged. Custom OCR backends and built-in backends that support runtime tuning can read this value and deserialize the keys they care about. Keys unknown to the backend are silently ignored. This is the recommended extension point for per-call parameters that are not covered by the typed fields above (e.g. mode switching, preprocessing flags, inference batch size). Scope: when pipeline is null, this value is propagated to the primary stage of the auto-constructed pipeline. When pipeline is explicitly set, this field has no effect — the caller must set OcrPipelineStage.backend_options directly on the relevant stage(s) instead. Example: json { "mode": "fast", "enable_layout": true, "timeout_ms": 5000 }
elementConfig	OcrElementConfig | null	null	OCR element extraction configuration
qualityThresholds	OcrQualityThresholds | null	null	Quality thresholds for the native-text-to-OCR fallback decision. When None, uses compiled defaults (matching previous hardcoded behavior).
pipeline	OcrPipelineConfig | null	null	Multi-backend OCR pipeline configuration. When set, enables weighted fallback across multiple OCR backends based on output quality. When None, uses the single backend field (same as today).
autoRotate	boolean	false	Enable automatic page rotation based on orientation detection. When enabled, uses Tesseract’s DetectOrientationScript() to detect page orientation (0/90/180/270 degrees) before OCR. If the page is rotated with high confidence, the image is corrected before recognition. This is critical for handling rotated scanned documents.
vlmFallback	VlmFallbackPolicy	VlmFallbackPolicy.Disabled	Ergonomic VLM fallback policy. When set to anything other than VlmFallbackPolicy.Disabled and OcrConfig.pipeline is null, a multi-stage pipeline is synthesised automatically: - VlmFallbackPolicy.OnLowQuality → \[classical_stage, vlm_stage\] with the quality_threshold mapped onto OcrQualityThresholds.pipeline_min_quality. - VlmFallbackPolicy.Always → \[vlm_stage\] only. Requires OcrConfig.vlm_config to be Some when not Disabled. When OcrConfig.pipeline is explicitly set, this field is ignored.
vlmConfig	LlmConfig | null	null	VLM (Vision Language Model) OCR configuration. Required when backend is "vlm" or when vlm_fallback is not VlmFallbackPolicy.Disabled. Uses liter-llm to send page images to a vision model for text extraction.
vlmPrompt	string | null	null	Custom Jinja2 prompt template for VLM OCR. When null, uses the default template. Available variables: - {{ language }} — The document language code (e.g., “eng”, “deu”).
acceleration	AccelerationConfig | null	null	Hardware acceleration for ONNX Runtime models (e.g. PaddleOCR, layout detection). Not user-configurable via config files — injected at runtime from ExtractionConfig.acceleration before each process_image call.
tessdataBytes	Record<string, Buffer> | null	null	Caller-supplied Tesseract traineddata bytes per language code. Primary use case is the WASM build, which has no filesystem and cannot download tessdata at runtime. Native builds typically rely on TessdataManager and ignore this field. When present, the WASM Tesseract backend prefers these bytes over its compile-time-bundled English data. Skipped by serde to keep config files small — supply via the typed API at runtime.
tessdataPath	string | null	null	Runtime override for tessdata directory path. When set, uses this path as the highest-priority tessdata location, bypassing environment variables and cache directories. Useful for embedding pre-installed tessdata in applications. When null, uses the standard resolution chain: TESSDATA_PREFIX env, cache dir, system paths.

Methods

default()

Signature:

static default(): OcrConfig

Example:

const result = OcrConfig.default();

Returns: OcrConfig

---

OcrElement

A unified OCR element representing detected text with full metadata.

This is the primary type for structured OCR output, preserving all information from both Tesseract and PaddleOCR backends.

Field	Type	Default	Description
text	string	—	The recognized text content.
geometry	OcrBoundingGeometry	OcrBoundingGeometry.Rectangle	Bounding geometry (rectangle or quadrilateral).
confidence	OcrConfidence	—	Confidence scores for detection and recognition.
level	OcrElementLevel	OcrElementLevel.Line	Hierarchical level (word, line, block, page).
rotation	OcrRotation | null	null	Rotation information (if detected).
pageNumber	number	—	Page number (1-indexed).
parentId	string | null	null	Parent element ID for hierarchical relationships. Only used for Tesseract output which has word -> line -> block hierarchy.
backendMetadata	Record<string, unknown>	{}	Backend-specific metadata that doesn’t fit the unified schema.

---

OcrElementConfig

Configuration for OCR element extraction.

Controls how OCR elements are extracted and filtered.

Field	Type	Default	Description
includeElements	boolean	—	Whether to include OCR elements in the extraction result. When true, the ocr_elements field in ExtractedDocument will be populated.
minLevel	OcrElementLevel	OcrElementLevel.Line	Minimum hierarchical level to include. Elements below this level (e.g., words when min_level is Line) will be excluded.
minConfidence	number	—	Minimum recognition confidence threshold (0.0-1.0). Elements with confidence below this threshold will be filtered out.
buildHierarchy	boolean	—	Whether to build hierarchical relationships between elements. When true, parent_id fields will be populated based on spatial containment. Only meaningful for Tesseract output.

---

OcrExtractionResult

OCR extraction result.

Result of performing OCR on an image or scanned document, including recognized text and detected tables.

Field	Type	Default	Description
content	string	—	Recognized text content
mimeType	string	—	Original MIME type of the processed image
metadata	Record<string, unknown>	{}	OCR processing metadata (confidence scores, language, etc.)
tables	Array<OcrTable>	\[\]	Tables detected and extracted via OCR
ocrElements	Array<OcrElement> | null	\[\]	Structured OCR elements with bounding boxes and confidence scores. Available when TSV output is requested or table detection is enabled.

---

OcrMetadata

OCR processing metadata.

Captures information about OCR processing configuration and results.

Field	Type	Default	Description
language	string	—	OCR language code(s) used
psm	number	—	Tesseract Page Segmentation Mode (PSM)
outputFormat	string	—	Output format (e.g., “text”, “hocr”)
tableCount	number	—	Number of tables detected
tableRows	number | null	null	Number of rows in the detected table (if a single table was found).
tableCols	number | null	null	Number of columns in the detected table (if a single table was found).

---

OcrPipelineConfig

Multi-backend OCR pipeline with quality-based fallback.

Backends are tried in priority order (highest first). After each backend produces output, quality is evaluated. If it meets quality_thresholds.pipeline_min_quality, the result is accepted. Otherwise the next backend is tried; if none clears the threshold, an internal selection policy derived from the OcrConfig decides which stage’s result is returned as the best effort (vlm_fallback pipelines prefer their last non-empty stage; explicit and classical pipelines stay score-based).

Field	Type	Default	Description
stages	Array<OcrPipelineStage>	—	Ordered list of backends to try. Sorted by priority (descending) at runtime.
qualityThresholds	OcrQualityThresholds	/* serde(default) */	Quality thresholds for deciding whether to accept a result or try the next backend.

---

OcrPipelineStage

A single backend stage in the OCR pipeline.

Field	Type	Default	Description
backend	string	—	Backend name: “tesseract”, “paddleocr”, “paddle-ocr”, “sceptre”, “vlm”, or a custom registered name. Sceptre uses ONNX Runtime on desktop/server and tract on Android/iOS; browser WebAssembly has a separate byte-fed engine because the normal async OCR registry assumes native model storage.
priority	number	serde(default = "default_priority")	Priority weight (higher = tried first). Stages are sorted by priority descending.
language	Array<string> | null	/* serde(default) */	Language override for this stage (None = use parent OcrConfig.language). A list is the canonical form and the only form accepted by the binding object APIs: \["eng", "deu"\]. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”).
tesseractConfig	TesseractConfig | null	/* serde(default) */	Tesseract-specific config override for this stage.
paddleOcrConfig	unknown | null	/* serde(default) */	PaddleOCR-specific config for this stage.
vlmConfig	LlmConfig | null	/* serde(default) */	VLM config override for this pipeline stage.
backendOptions	unknown | null	/* serde(default) */	Arbitrary per-call options passed through to the backend unchanged. Backends that support runtime tuning (mode switching, preprocessing flags, inference parameters, etc.) read this value and deserialize the keys they care about. Keys unknown to the backend are silently ignored, so options from different backends can coexist in the same config without conflict. Example (custom backend): json { "mode": "fast", "enable_layout": true }

---

OcrQualityThresholds

Quality thresholds for OCR fallback decisions and pipeline quality gating.

All fields default to the values that match the previous hardcoded behavior, so OcrQualityThresholds.default() preserves existing semantics exactly.

Field	Type	Default	Description
minTotalNonWhitespace	number	64	Minimum total non-whitespace characters to consider text substantive.
minNonWhitespacePerPage	number	32	Minimum non-whitespace characters per page on average.
minMeaningfulWordLen	number	4	Minimum character count for a word to be “meaningful”.
minMeaningfulWords	number	3	Minimum count of meaningful words before text is accepted.
minAlnumRatio	number	0.3	Minimum alphanumeric ratio (non-whitespace chars that are alphanumeric).
minGarbageChars	number	5	Minimum Unicode replacement characters (U+FFFD) to trigger OCR fallback.
maxFragmentedWordRatio	number	0.6	Maximum fraction of short (1-2 char) words before text is considered fragmented.
criticalFragmentedWordRatio	number	0.8	Critical fragmentation threshold — triggers OCR regardless of meaningful words. Normal English text has ~20-30% short words. 80%+ is definitive garbage.
minAvgWordLength	number	2	Minimum average word length. Below this with enough words indicates garbled extraction.
minWordsForAvgLengthCheck	number	50	Minimum word count before average word length check applies.
minConsecutiveRepeatRatio	number	0.08	Minimum consecutive word repetition ratio to detect column scrambling.
minWordsForRepeatCheck	number	50	Minimum word count before consecutive repetition check is applied.
substantiveMinChars	number	100	Minimum character count for “substantive markdown” OCR skip gate.
nonTextMinChars	number	20	Minimum character count for “non-text content” OCR skip gate.
alnumWsRatioThreshold	number	0.4	Alphanumeric+whitespace ratio threshold for skip decisions.
pipelineMinQuality	number	0.5	Minimum quality score (0.0-1.0) for a pipeline stage result to be accepted. If the result from a backend scores below this, try the next backend.
minUndecodableRatio	number	—	Minimum fraction of non-whitespace characters that are undecodable (Unicode Private Use Area, replacement characters, or non-whitespace control characters) before a page’s text layer is treated as unreadable and routed to OCR (issue #1254). Gated by min_total_non_whitespace so short snippets with a stray symbol or two do not trip this check.
enableProvenanceOcrRouting	boolean	—	Whether to route a page to OCR when pdf_oxide reports that a high fraction of its text was fabricated rather than read from the file (MappingProvenance.Fallback, pdf_oxide 0.3.75+, issue #1254). This is a direct fact from the extractor’s ISO 32000-1 §9.10.2 mapping cascade, distinct from the character-heuristic proxy behind min_undecodable_ratio. Defaults to true.
minProvenanceFallbackRatio	number	—	Minimum fraction of a page’s non-whitespace characters with MappingProvenance.Fallback provenance before the page is treated as having a fabricated text layer and routed to OCR (issue #1254). Gated by min_total_non_whitespace so a short page with a stray fallback character cannot trip it. Only used when enable_provenance_ocr_routing is true.

Methods

default()

Signature:

static default(): OcrQualityThresholds

Example:

const result = OcrQualityThresholds.default();

Returns: OcrQualityThresholds

---

OcrRotation

Rotation information for an OCR element.

Field	Type	Default	Description
angleDegrees	number	—	Rotation angle in degrees (0, 90, 180, 270 for PaddleOCR).
confidence	number | null	null	Confidence score for the rotation detection.

---

OcrTable

Table detected via OCR.

Represents a table structure recognized during OCR processing.

Field	Type	Default	Description
cells	Array<Array<string>>	—	Table cells as a 2D vector (rows × columns)
markdown	string	—	Markdown representation of the table
pageNumber	number	—	Page number where the table was found (1-indexed)
boundingBox	OcrTableBoundingBox | null	/* serde(default) */	Bounding box of the table in pixel coordinates (from OCR word positions).

---

OcrTableBoundingBox

Bounding box for an OCR-detected table in pixel coordinates.

Field	Type	Default	Description
left	number	—	Left x-coordinate (pixels)
top	number	—	Top y-coordinate (pixels)
right	number	—	Right x-coordinate (pixels)
bottom	number	—	Bottom y-coordinate (pixels)

---

OrientationResult

Document orientation detection result.

Field	Type	Default	Description
degrees	number	—	Detected orientation in degrees (0, 90, 180, or 270).
confidence	number	—	Confidence score (0.0-1.0).

---

PaddleOcrConfig

Configuration for PaddleOCR backend.

Configures PaddleOCR text detection and recognition with multi-language support. Uses a builder pattern for convenient configuration.

Field	Type	Default	Description
language	string	—	Language code (e.g., “en”, “ch”, “jpn”, “kor”, “deu”, “fra”)
cacheDir	string | null	null	Optional Hugging Face Hub cache root for model files. When unset, the standard HF_HUB_CACHE, legacy HUGGINGFACE_HUB_CACHE, and HF_HOME conventions are used.
useAngleCls	boolean	—	Enable angle classification for rotated text (default: false). Can misfire on short text regions, rotating crops incorrectly before recognition.
enableTableDetection	boolean	—	Enable table structure detection (default: false)
detDbThresh	number	—	Database threshold for text detection (default: 0.3) Range: 0.0-1.0, higher values require more confident detections
detDbBoxThresh	number	—	Box threshold for text bounding box refinement (default: 0.5) Range: 0.0-1.0
detDbUnclipRatio	number	—	Unclip ratio for expanding text bounding boxes (default: 1.6) Controls the expansion of detected text regions
detLimitSideLen	number	—	Maximum side length for detection image (default: 1024) Larger images may be resized to this limit for faster inference
recBatchNum	number	—	Batch size for recognition inference (default: 6) Number of text regions to process simultaneously
padding	number	—	Padding in pixels added around the image before detection (default: 10). Large values can include surrounding content like table gridlines.
dropScore	number	—	Minimum recognition confidence score for text lines (default: 0.5). Text regions with recognition confidence below this threshold are discarded. Matches PaddleOCR Python’s drop_score parameter. Range: 0.0-1.0
modelTier	string	—	Model tier controlling detection/recognition model size and accuracy trade-off. For PP-OCRv5 (model_version = "pp-ocrv5"): - "mobile" (default): Lightweight models (~4.5MB detection, ~16.5MB recognition), fast download and inference - "server": Large, high-accuracy models (~88MB detection, ~84MB recognition), best for GPU or complex documents For PP-OCRv6 (model_version = "pp-ocrv6"): "medium" (default), "small", or "tiny". A legacy "mobile"/"server" tier under v6 falls back to "medium".
modelVersion	string	—	Model generation: "pp-ocrv6" (default) or "pp-ocrv5". PP-OCRv6 adds a unified CJK+Latin+JA/KO recognition model with medium/small/tiny tiers (see model_tier). Scripts outside the v6 unified coverage (Arabic, Cyrillic, Devanagari, Greek, Tamil, Telugu, Thai) transparently fall back to the PP-OCRv5 per-script recognition models. Defaults to "pp-ocrv6"; the default model_tier ("mobile") resolves to the v6 "medium" tier. Select "pp-ocrv5" to pin the legacy per-script/unified fleet.
inferenceBackend	PaddleInferenceBackend | null	null	Explicit inference engine choice. null (the default) resolves to the compiled default: ort when the paddle-ocr-ort feature is compiled in, otherwise tract. An explicit choice is validated against the compiled features when the OCR engine is constructed (see crate.paddle_ocr.backend.effective_backend); requesting an engine whose feature is not compiled in is a clear configuration error rather than a silent fallback.

Methods

withCacheDir()

Sets a custom Hugging Face Hub cache root for model files.

Signature:

withCacheDir(path: string): PaddleOcrConfig

Example:

const result = instance.withCacheDir("value");

Parameters:

Name	Type	Required	Description
path	string	Yes	Path to cache directory

Returns: PaddleOcrConfig

withTableDetection()

Enables or disables table structure detection.

Signature:

withTableDetection(enable: boolean): PaddleOcrConfig

Example:

const result = instance.withTableDetection(true);

Parameters:

Name	Type	Required	Description
enable	boolean	Yes	Whether to enable table detection

Returns: PaddleOcrConfig

withAngleCls()

Enables or disables angle classification for rotated text.

Signature:

withAngleCls(enable: boolean): PaddleOcrConfig

Example:

const result = instance.withAngleCls(true);

Parameters:

Name	Type	Required	Description
enable	boolean	Yes	Whether to enable angle classification

Returns: PaddleOcrConfig

withDetDbThresh()

Sets the database threshold for text detection.

Signature:

withDetDbThresh(threshold: number): PaddleOcrConfig

Example:

const result = instance.withDetDbThresh(0.5);

Parameters:

Name	Type	Required	Description
threshold	number	Yes	Detection threshold (0.0-1.0)

Returns: PaddleOcrConfig

withDetDbBoxThresh()

Sets the box threshold for text bounding box refinement.

Signature:

withDetDbBoxThresh(threshold: number): PaddleOcrConfig

Example:

const result = instance.withDetDbBoxThresh(0.5);

Parameters:

Name	Type	Required	Description
threshold	number	Yes	Box threshold (0.0-1.0)

Returns: PaddleOcrConfig

withDetDbUnclipRatio()

Sets the unclip ratio for expanding text bounding boxes.

Signature:

withDetDbUnclipRatio(ratio: number): PaddleOcrConfig

Example:

const result = instance.withDetDbUnclipRatio(0.5);

Parameters:

Name	Type	Required	Description
ratio	number	Yes	Unclip ratio (typically 1.5-2.0)

Returns: PaddleOcrConfig

withDetLimitSideLen()

Sets the maximum side length for detection images.

Signature:

withDetLimitSideLen(length: number): PaddleOcrConfig

Example:

const result = instance.withDetLimitSideLen(42);

Parameters:

Name	Type	Required	Description
length	number	Yes	Maximum side length in pixels

Returns: PaddleOcrConfig

withRecBatchNum()

Sets the batch size for recognition inference.

Signature:

withRecBatchNum(batchSize: number): PaddleOcrConfig

Example:

const result = instance.withRecBatchNum(42);

Parameters:

Name	Type	Required	Description
batchSize	number	Yes	Number of text regions to process simultaneously

Returns: PaddleOcrConfig

withDropScore()

Sets the minimum recognition confidence threshold.

Signature:

withDropScore(score: number): PaddleOcrConfig

Example:

const result = instance.withDropScore(0.5);

Parameters:

Name	Type	Required	Description
score	number	Yes	Minimum confidence (0.0-1.0), text below this is dropped

Returns: PaddleOcrConfig

withPadding()

Sets padding in pixels added around images before detection.

Signature:

withPadding(padding: number): PaddleOcrConfig

Example:

const result = instance.withPadding(42);

Parameters:

Name	Type	Required	Description
padding	number	Yes	Padding in pixels (0-100)

Returns: PaddleOcrConfig

withModelTier()

Sets the model tier controlling detection/recognition model size.

Signature:

withModelTier(tier: string): PaddleOcrConfig

Example:

const result = instance.withModelTier("value");

Parameters:

Name	Type	Required	Description
tier	string	Yes	"mobile" (default, lightweight, faster) or "server" (high accuracy, GPU/complex documents)

Returns: PaddleOcrConfig

withModelVersion()

Sets the model generation.

selects among "medium"/"small"/"tiny".

Signature:

withModelVersion(version: string): PaddleOcrConfig

Example:

const result = instance.withModelVersion("value");

Parameters:

Name	Type	Required	Description
version	string	Yes	"pp-ocrv6" (default) or "pp-ocrv5". Under "pp-ocrv6", model_tier

Returns: PaddleOcrConfig

default()

Creates a default configuration with English language support.

Signature:

static default(): PaddleOcrConfig

Example:

const result = PaddleOcrConfig.default();

Returns: PaddleOcrConfig

---

PageBoundary

Byte offset boundary for a page.

Tracks where a specific page’s content starts and ends in the main content string, enabling mapping from byte positions to page numbers. Offsets are guaranteed to be at valid UTF-8 character boundaries when using standard String methods (push_str, push, etc.).

Field	Type	Default	Description
byteStart	number	—	Byte offset where this page starts in the content string (UTF-8 valid boundary, inclusive)
byteEnd	number	—	Byte offset where this page ends in the content string (UTF-8 valid boundary, exclusive)
pageNumber	number	—	Page number (1-indexed)

---

PageClassification

Classification result for a single page.

Field	Type	Default	Description
pageNumber	number	—	1-indexed page number this classification belongs to.
labels	Array<ClassificationLabel>	—	Labels assigned to the page. Single-label classification yields exactly one entry; multi-label classification yields any subset of the configured label set.

---

PageClassificationConfig

Since: v1.0

Configuration for the page-classification post-processor.

Field	Type	Default	Description
promptTemplate	string | null	null	Minijinja prompt template. Receives {{ labels }} (joined list), {{ page_text }} and {{ multi_label }} variables. null lets the backend pick a sensible default.
labels	Array<string>	—	The set of labels the classifier may emit. Must contain at least one entry.
multiLabel	boolean	/* serde(default) */	Allow multiple labels per page. Single-label mode returns at most one label.
llm	LlmConfig	—	LLM configuration used for classification.

---

PageConfig

Page extraction and tracking configuration.

Controls how pages are extracted, tracked, and represented in the extraction results. When null, page tracking is disabled.

Page range tracking in chunk metadata (first_page/last_page) is automatically enabled when page boundaries are available and chunking is configured.

Field	Type	Default	Description
extractPages	boolean	false	Extract pages as separate array (ExtractedDocument.pages)
insertPageMarkers	boolean	false	Insert page markers in main content string
markerFormat	string	"<!-- PAGE {page_num} -->"	Page marker format (use {page_num} placeholder) Default: “\n\n\n\n”

Methods

default()

Signature:

static default(): PageConfig

Example:

const result = PageConfig.default();

Returns: PageConfig

---

PageContent

Content for a single page/slide.

When page extraction is enabled, documents are split into per-page content with associated tables and images mapped to each page.

Performance

Uses shared tables and images for memory efficiency:

• Table[] enables zero-copy sharing of table data
• ExtractedImage[] enables zero-copy sharing of image data
• Maintains exact JSON compatibility via custom Serialize/Deserialize

This reduces memory overhead for documents with shared tables/images by avoiding redundant copies during serialization.

Field	Type	Default	Description
pageNumber	number	—	Page number (1-indexed)
content	string	—	Text content for this page
tables	Array<Table>	/* serde(default) */	Tables found on this page (uses Arc for memory efficiency) Serializes as Table\[\] for JSON compatibility while maintaining shared in-memory ownership for zero-copy sharing.
imageIndices	Array<number>	/* serde(default) */	Indices into ExtractedDocument.images for images found on this page. Each value is a zero-based index into the top-level images collection. Only populated when extract_images = true in the extraction config.
hierarchy	PageHierarchy | null	null	Hierarchy information for the page (when hierarchy extraction is enabled) Contains text hierarchy levels (H1-H6) extracted from the page content.
isBlank	boolean | null	null	Whether this page is blank (no meaningful text content) Determined during extraction based on text content analysis. A page is blank if it has fewer than 3 non-whitespace characters and contains no tables or images.
layoutRegions	Array<LayoutRegion> | null	null	Layout detection regions for this page (when layout detection is enabled). Contains detected layout regions with class, confidence, bounding box, and area fraction. Only populated when layout detection is configured.
speakerNotes	string | null	null	Speaker notes for this slide (PPTX only). Contains the text from the slide’s notes pane (ppt/notesSlides/notesSlide{N}.xml). Only populated when the source is a PPTX file and notes are present.
sectionName	string | null	null	Section name this slide belongs to (PPTX only). PowerPoint sections group slides into logical chapters (<p:sectionLst> in ppt/presentation.xml). Only populated when the source is a PPTX file and the slide belongs to a named section.
sheetName	string | null	null	Sheet name for this page (XLSX/ODS only). Each spreadsheet sheet maps to one PageContent entry. This field carries the sheet’s display name as it appears in the workbook. null for all non-spreadsheet formats and for sheets with an empty name.

---

PageHierarchy

Page hierarchy structure containing heading levels and block information.

Used when PDF text hierarchy extraction is enabled. Contains hierarchical blocks with heading levels (H1-H6) for semantic document structure.

Field	Type	Default	Description
blockCount	number	—	Number of hierarchy blocks on this page
blocks	Array<HierarchicalBlock>	/* serde(default) */	Hierarchical blocks with heading levels

---

PageInfo

Metadata for individual page/slide/sheet.

Captures per-page information including dimensions, content counts, and visibility state (for presentations).

Field	Type	Default	Description
number	number	—	Page number (1-indexed)
title	string | null	null	Page title (usually for presentations)
imageCount	number | null	null	Number of images on this page
tableCount	number | null	null	Number of tables on this page
hidden	boolean | null	null	Whether this page is hidden (e.g., in presentations)
isBlank	boolean | null	null	Whether this page is blank (no meaningful text, no images, no tables) A page is considered blank if it has fewer than 3 non-whitespace characters and contains no tables or images. This is useful for filtering out empty pages in scanned documents or PDFs with blank separator pages.
hasVectorGraphics	boolean	/* serde(default) */	Whether this page contains non-trivial vector graphics (paths, shapes, curves) Indicates the presence of vector-drawn content such as charts, diagrams, or geometric shapes (e.g., from Adobe InDesign, LaTeX TikZ). These are invisible to ExtractedDocument.images since they are not embedded as raster XObjects. Set to true when path count exceeds a heuristic threshold, signaling that downstream consumers may want to rasterize the page to capture this content. Only populated for PDFs; null for other document types.

---

PageRange

Page range for a chunk (0-indexed, inclusive).

Field	Type	Default	Description
start	number	—	Start page (0-indexed, inclusive).
end	number	—	End page (0-indexed, inclusive).

Methods

pageCount()

Get the number of pages in this range.

Signature:

pageCount(): number

Example:

const result = instance.pageCount();

Returns: number

---

PageSignals

Per-page signals extracted from PDF content.

Field	Type	Default	Description
pageNumber	number	—	1-indexed page number.
textExcerpt	string	—	First ~500 characters of extracted text.
startsWithLetterheadLike	boolean	—	true if page starts with letterhead-like content (ALL CAPS line in first 5 lines or a logo-image bbox at top).
hasPageNumberOneMarker	boolean	—	true if text contains “Page 1” or “1 of N” pattern.
hasSignatureBlock	boolean	—	true if text contains signature indicators (“Sincerely”, “Signed”) or a signature image bbox.
layoutTextDensity	number	—	Text density: characters per page area, normalised to \[0.0, 1.0\].

Methods

fromPageText()

Derive signals from raw page text.

Callers that already have structured per-page data (e.g. from a PDF extractor) can set individual fields directly. This constructor is for callers that only have the plain-text content of a page (e.g. from PageContent).

when unknown (disables density-shift detection for this page).

Heuristics

All signal derivations are conservative starting points. Each is documented inline. They err on the side of fewer false positives; tune thresholds via MultidocThresholds rather than by changing these heuristics.

Signature:

static fromPageText(pageNumber: number, text: string, layoutTextDensity: number): PageSignals

Example:

const result = PageSignals.fromPageText(42, "value", 0.5);

Parameters:

Name	Type	Required	Description
pageNumber	number	Yes	The page number
text	string	Yes	The text
layoutTextDensity	number	Yes	The layout text density

Returns: PageSignals

---

PageSpan

A single page covered by a chunk, with an optional bounding box on that page.

See ChunkMetadata.page_spans (#1295) for population semantics.

Field	Type	Default	Description
page	number	—	Page number (1-indexed).
bbox	BoundingBox | null	null	Bounding box on this page, if known.

---

PageStructure

Unified page structure for documents.

Supports different page types (PDF pages, PPTX slides, Excel sheets) with character offset boundaries for chunk-to-page mapping.

Field	Type	Default	Description
totalCount	number	—	Total number of pages/slides/sheets
unitType	PageUnitType	—	Type of paginated unit
boundaries	Array<PageBoundary> | null	null	Character offset boundaries for each page Maps character ranges in the extracted content to page numbers. Used for chunk page range calculation.
pages	Array<PageInfo> | null	null	Detailed per-page metadata (optional, only when needed)

---

PatternMatch

One detected PII span in the input text.

Field	Type	Default	Description
start	number	—	Inclusive byte-offset start of the match in the source text.
end	number	—	Exclusive byte-offset end of the match.
category	PiiCategory	—	Category the match belongs to.
text	string	—	Matched substring (owned copy — pattern engine returns owned data so the caller can free the original text if needed before replacement).

---

PdfAnnotation

A PDF annotation extracted from a document page.

Field	Type	Default	Description
annotationType	PdfAnnotationType	—	The type of annotation.
content	string | null	null	Text content of the annotation (e.g., comment text, link URL).
pageNumber	number	—	Page number where the annotation appears (1-indexed).
boundingBox	BoundingBox | null	null	Bounding box of the annotation on the page.
author	string | null	/* serde(default) */	Author/creator of the annotation (PDF /T entry).
modified	string | null	/* serde(default) */	Last modification date of the annotation (PDF /M entry), as a raw PDF date string (e.g. "D:20240115120000Z").
color	string | null	/* serde(default) */	Annotation colour (PDF /C entry), normalised to a CSS-compatible #rrggbb hex string. Gray and CMYK colour spaces are converted to RGB.
subject	string | null	/* serde(default) */	Subject of the annotation (PDF /Subj entry).
quadPoints	Array<BoundingBox> | null	/* serde(default) */	Per-line bounding boxes derived from the annotation’s /QuadPoints entry. Present for text markup annotations (Highlight, Underline, StrikeOut, Squiggly), one box per marked line/run of text.
markedText	string | null	/* serde(default) */	The document text covered by Self.quad_points, recovered from the page content underneath the marked-up region. Populated for Highlight, Underline, StrikeOut, and Squiggly annotations when the underlying text could be recovered.

---

PdfConfig

PDF-specific configuration.

Field	Type	Default	Description
extractImages	boolean	false	Extract images from PDF
extractTables	boolean	true	Extract tables from PDF. When true (default), runs pdf_oxide’s native grid detector and, if it finds nothing, falls back to the heuristic text-layer reconstruction in pdf.oxide.table.extract_tables_heuristic. Set to false to skip both passes — tables will then be empty in the result.
passwords	Array<string> | null	null	List of passwords to try when opening encrypted PDFs
extractMetadata	boolean	true	Extract PDF metadata
hierarchy	HierarchyConfig | null	null	Hierarchy extraction configuration (None = hierarchy extraction disabled)
extractAnnotations	boolean	false	Extract PDF annotations (text notes, highlights, links, stamps). Default: false
topMarginFraction	number | null	null	Top margin fraction (0.0–1.0) of page height to exclude headers/running heads. Default: 0.06 (6%)
bottomMarginFraction	number | null	null	Bottom margin fraction (0.0–1.0) of page height to exclude footers/page numbers. Default: 0.05 (5%)
allowSingleColumnTables	boolean	false	Allow single-column pseudo tables in extraction results. By default, tables with fewer than 2 columns (layout-guided) or 3 columns (heuristic) are rejected. When true, the minimum column count is relaxed to 1, allowing single-column structured data (glossaries, itemized lists) to be emitted as tables. Other quality filters (density, sparsity, prose detection) still apply.
ocrInlineImages	boolean	false	Perform OCR on inline images extracted from PDF pages and attach the recognized text to each ExtractedImage.ocr_result. Requires Tesseract to be available; if ExtractionConfig.ocr is null the extractor falls back to TesseractConfig.default(). Per-image failures degrade gracefully (the image is returned without OCR text rather than failing the whole extraction). Default: false.
extractFormFields	boolean	true	Extract AcroForm and XFA form fields into ExtractedDocument.form_fields. When true (default), reads the document’s interactive form structure (field names, types, values, widget geometry). Cheap and strictly additive — non-form PDFs simply yield an empty list. Set to false to skip the form pass entirely.
readingOrder	boolean	false	Reorder extracted text by layout-detected reading order. When true, projects text spans onto layout-detected regions, performs column detection, and emits spans in natural reading order (important for multi-column academic PDFs). It also repairs 90/180/270-degree rotated text runs — sideways tables and captions — that otherwise read word-reversed and glued (GH#1358); see crate.extractors.pdf.reading_order for the rotation-handling details and its limits. Requires the layout-detection feature and a page for which layout detection actually produces hints: a page with no detected regions falls back to the original, unrepaired extraction order even with this enabled. Independent of LayoutStrategy, which only controls whether layout detection runs at all — enabling Always or Auto alone does not turn reordering on. Defaults to false.

Methods

default()

Signature:

static default(): PdfConfig

Example:

const result = PdfConfig.default();

Returns: PdfConfig

---

PdfFormField

A form field extracted from a PDF’s AcroForm or XFA structure.

Populated by the PDF extractor when PdfConfig.extract_form_fields is enabled and the document is a fillable form. Supports both AcroForm (standard) and XFA (XML Forms Architecture) layers. When both are present, AcroForm fields take priority (canonical fallback per PDF spec), and XFA-only fields are appended. The collection is empty for non-form PDFs and for non-PDF formats.

PdfConfig.extract_form_fields: crate.core.config.PdfConfig.extract_form_fields

Field	Type	Default	Description
name	string	—	Partial field name (the leaf name within the field hierarchy).
fullName	string	—	Fully-qualified field name (dotted path from the form root).
fieldType	FormFieldType	—	Classified field type.
value	string | null	/* serde(default) */	Current field value, if any.
defaultValue	string | null	/* serde(default) */	Default field value, if any.
flags	number	/* serde(default) */	Raw field-flags bitmask (read-only, required, multiline, …).
page	number | null	/* serde(default) */	1-indexed page the field’s widget appears on. Currently always null for AcroForm fields; page assignment is a deferred enhancement requiring spatial analysis of widget annotations per page.
bbox	BoundingBox | null	/* serde(default) */	Widget bounding box on its page, if known.
maxLength	number | null	/* serde(default) */	Maximum input length for text fields, if specified.
tooltip	string | null	/* serde(default) */	Tooltip / alternate field description, if present.

---

PdfMetadata

PDF-specific metadata.

Contains metadata fields specific to PDF documents that are not in the common Metadata structure. Common fields like title, authors, keywords, and dates are at the Metadata level.

Field	Type	Default	Description
pdfVersion	string | null	null	PDF version (e.g., “1.7”, “2.0”)
producer	string | null	null	PDF producer (application that created the PDF)
isEncrypted	boolean | null	null	Whether the PDF is encrypted/password-protected
width	number | null	null	First page width in points (1/72 inch)
height	number | null	null	First page height in points (1/72 inch)
pageCount	number | null	null	Total number of pages in the PDF document
scannedConfidence	number | null	null	How strongly the document’s most scan-like page resembles a scan, in \[0.0, 1.0\]. null when the document could not be inspected. A full-page raster with no visible text scores at least 0.85; a born-digital slide with a full-bleed background image scores 0.50.
scannedPages	Array<number> | null	\[\]	Pages that look like scans (1-indexed), using the default confidence threshold. null when the document could not be inspected; empty when no page qualifies.
layoutGatedPages	Array<number> | null	\[\]	Pages the auto layout strategy skipped (1-indexed). null unless layout detection ran with LayoutStrategy.Auto; empty when the gate selected every page.
layoutGateReasons	Array<string> | null	\[\]	Why the auto layout gate selected or skipped each page. Index i is page i + 1. Snake_case values such as multi_column, table_grid, or plain_text (the skip reason). null unless layout detection ran with LayoutStrategy.Auto.

---

Plugin

Base trait that all plugins must implement.

This trait provides common functionality for plugin lifecycle management, identification, and metadata.

Thread Safety

All plugins must be Send + Sync to support concurrent usage across threads.

Methods

name()

Returns the unique name/identifier for this plugin.

The name should be:

• Unique across all plugins
• Lowercase with hyphens (e.g., “my-custom-plugin”)
• URL-safe characters only

Signature:

name(): string

Example:

const result = instance.name();

Returns: string

version()

Returns the semantic version of this plugin.

Should follow semver format: MAJOR.MINOR.PATCH

Defaults to the xberg crate version.

Signature:

version(): string

Example:

const result = instance.version();

Returns: string

initialize()

Initialize the plugin.

Called once when the plugin is registered. Use this to:

• Load configuration
• Initialize resources (connections, caches, etc.)
• Validate dependencies

Thread Safety

This method takes &self instead of &mut self to work with Arc<dyn Plugin>. Plugins needing mutable state during initialization should use interior mutability patterns (Mutex, RwLock, OnceCell, etc.).

Errors:

Should return an error if initialization fails. The plugin will not be registered if this method returns an error.

Defaults to a no-op for stateless plugins.

Signature:

initialize(): void

Example:

instance.initialize();

Returns: No return value.

Errors: Throws Error with a descriptive message.

shutdown()

Shutdown the plugin.

Called when the plugin is being unregistered or the application is shutting down. Use this to:

• Close connections
• Flush caches
• Release resources

Thread Safety

This method takes &self instead of &mut self to work with Arc<dyn Plugin>. Plugins needing mutable state during shutdown should use interior mutability patterns (Mutex, RwLock, etc.).

Errors:

Errors during shutdown are logged but don’t prevent the shutdown process.

Defaults to a no-op for stateless plugins.

Signature:

shutdown(): void

Example:

instance.shutdown();

Returns: No return value.

Errors: Throws Error with a descriptive message.

description()

Optional plugin description for debugging and logging.

Defaults to empty string if not overridden.

Signature:

description(): string

Example:

const result = instance.description();

Returns: string

author()

Optional plugin author information.

Defaults to empty string if not overridden.

Signature:

author(): string

Example:

const result = instance.author();

Returns: string

---

PostProcessor

Trait for post-processor plugins.

Post-processors transform or enrich extraction results after the initial extraction is complete. They can:

• Clean and normalize text
• Add metadata (language, keywords, entities)
• Split content into chunks
• Score quality
• Apply custom transformations

Processing Order

Post-processors are executed in stage order:

1. Early - Language detection, entity extraction
2. Middle - Keyword extraction, token reduction
3. Late - Custom hooks, final validation

Within each stage, processors are executed in registration order.

Error Handling

Post-processor errors are non-fatal by default - they’re captured in metadata and execution continues. To make errors fatal, return an error from process().

Thread Safety

Post-processors must be thread-safe (Send + Sync).

Methods

process()

Process an extraction result.

Transform or enrich the extraction result. Can modify:

• content - The extracted text
• metadata - Add or update metadata fields
• tables - Modify or enhance table data

Returns:

Ok(()) if processing succeeded, Err(...) for fatal failures.

Errors:

Return errors for fatal processing failures. Non-fatal errors should be captured in metadata directly on the result.

Performance

This signature avoids unnecessary cloning of large extraction results by taking a mutable reference instead of ownership. Processors modify the result in place.

Example - Language Detection

Example - Text Cleaning

Signature:

process(result: ExtractedDocument, config: ExtractionConfig): Promise<void>

Example:

await instance.process(new ExtractedDocument(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	Mutable reference to the extraction result to process
config	ExtractionConfig	Yes	Extraction configuration

Returns: No return value.

Errors: Throws Error with a descriptive message.

processingStage()

Get the processing stage for this post-processor.

Determines when this processor runs in the pipeline.

Returns:

The ProcessingStage (Early, Middle, or Late).

Signature:

processingStage(): ProcessingStage

Example:

const result = instance.processingStage();

Returns: ProcessingStage

shouldProcess()

Optional: Check if this processor should run for a given result.

Allows conditional processing based on MIME type, metadata, or content. Defaults to true (always run).

Returns:

true if the processor should run, false to skip.

Signature:

shouldProcess(result: ExtractedDocument, config: ExtractionConfig): boolean

Example:

const result = instance.shouldProcess(new ExtractedDocument(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document
config	ExtractionConfig	Yes	The extraction config

Returns: boolean

estimatedDurationMs()

Optional: Estimate processing time in milliseconds.

Used for logging and debugging. Defaults to 0 (unknown).

Returns:

Estimated processing time in milliseconds.

Signature:

estimatedDurationMs(result: ExtractedDocument): number

Example:

const result = instance.estimatedDurationMs(new ExtractedDocument());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document

Returns: number

priority()

Execution priority within the processing stage.

Higher values run first within the same ProcessingStage. Defaults to 50. Use 0-49 for fallback processors, 50 for normal processors, and 51-255 for high-priority processors that should run early in their stage.

Signature:

priority(): number

Example:

const result = instance.priority();

Returns: number

---

PostProcessorConfig

Post-processor configuration.

Field	Type	Default	Description
enabled	boolean	true	Enable post-processors
enabledProcessors	Array<string> | null	null	Whitelist of processor names to run (None = all enabled)
disabledProcessors	Array<string> | null	null	Blacklist of processor names to skip (None = none disabled)
enabledSet	Array<string> | null	null	Pre-computed AHashSet for O(1) enabled processor lookup
disabledSet	Array<string> | null	null	Pre-computed AHashSet for O(1) disabled processor lookup

Methods

default()

Signature:

static default(): PostProcessorConfig

Example:

const result = PostProcessorConfig.default();

Returns: PostProcessorConfig

---

PptxAppProperties

Application properties from docProps/app.xml for PPTX

Contains PowerPoint-specific document metadata.

Field	Type	Default	Description
application	string | null	null	Application name (e.g., “Microsoft Office PowerPoint”)
appVersion	string | null	null	Application version
totalTime	number | null	null	Total editing time in minutes
company	string | null	null	Company name
docSecurity	number | null	null	Document security level
scaleCrop	boolean | null	null	Scale crop flag
linksUpToDate	boolean | null	null	Links up to date flag
sharedDoc	boolean | null	null	Shared document flag
hyperlinksChanged	boolean | null	null	Hyperlinks changed flag
slides	number | null	null	Number of slides
notes	number | null	null	Number of notes
hiddenSlides	number | null	null	Number of hidden slides
multimediaClips	number | null	null	Number of multimedia clips
presentationFormat	string | null	null	Presentation format (e.g., “Widescreen”, “Standard”)
slideTitles	Array<string>	\[\]	Slide titles

---

PptxExtractionResult

PowerPoint (PPTX) extraction result.

Contains extracted slide content, metadata, and embedded images/tables.

Field	Type	Default	Description
content	string	—	Extracted text content from all slides
metadata	PptxMetadata	—	Presentation metadata
slideCount	number	—	Total number of slides
imageCount	number	—	Total number of embedded images
tableCount	number	—	Total number of tables
images	Array<ExtractedImage>	—	Extracted images from the presentation
pageStructure	PageStructure | null	null	Slide structure with boundaries (when page tracking is enabled)
pageContents	Array<PageContent> | null	null	Per-slide content (when page tracking is enabled)
document	DocumentStructure | null	null	Structured document representation
officeMetadata	Record<string, string>	/* serde(default) */	Office metadata extracted from docProps/core.xml and docProps/app.xml. Contains keys like “title”, “author”, “created_by”, “subject”, “keywords”, “modified_by”, “created_at”, “modified_at”, etc.
revisions	Array<DocumentRevision> | null	/* serde(default) */	Slide comments as revisions. Each <p:cm> element in ppt/comments/comment{N}.xml becomes a DocumentRevision { kind: Comment } with author (resolved from ppt/commentAuthors.xml), ISO-8601 timestamp, and RevisionAnchor.Slide { index }. null when no comment XML parts exist.

---

PptxMetadata

PowerPoint presentation metadata.

Extracted from PPTX files containing slide counts and presentation details.

Field	Type	Default	Description
slideCount	number	—	Total number of slides in the presentation
slideNames	Array<string>	\[\]	Names of slides (if available)
imageCount	number | null	null	Number of embedded images
tableCount	number | null	null	Number of tables

---

PreprocessingOptions

HTML preprocessing options for document cleanup before conversion.

Field	Type	Default	Description
enabled	boolean	true	Enable HTML preprocessing globally
preset	PreprocessingPreset	PreprocessingPreset.Standard	Preprocessing preset level (Minimal, Standard, Aggressive)
removeNavigation	boolean	true	Remove navigation elements (nav, breadcrumbs, menus, sidebars)
removeForms	boolean	true	Remove form elements (forms, inputs, buttons, etc.)

---

Preset

A curated structured-extraction preset loaded from the embedded library.

Each preset is a JSON file under src/presets/library/<id>/v1.json that validates against the meta-schema in src/presets/preset.schema.json.

Downstream catalog consumers can inject presets via extend_from_dir. The embedded OSS library ships only the generic_document toy preset.

Field	Type	Default	Description
id	string	—	Stable, URL-safe preset identifier (lowercase snake_case).
version	string	—	Monotonic version string (e.g. v1).
schemaName	string	—	Human-readable schema name forwarded to the LLM as the response/tool name.
description	string	—	One-line preset description shown in the registry UI.
category	PresetCategory	—	Top-level category for grouping in the playground.
tags	Array<string>	/* serde(default) */	Free-form tags used for search/filtering. May be empty.
schema	unknown	—	JSON Schema (Draft 2020-12) describing the structured output shape.
systemPrompt	string	—	Instruction primer sent to the model.
contextTemplate	string | null	/* serde(default) */	Optional mustache-style template merged with caller-supplied context.
mergeMode	MergeMode	—	Strategy for merging per-batch outputs across paginated calls.
preferredCallMode	CallMode	—	Default call mode suggested for this preset; heuristics may override.
emitCitations	boolean	—	When true, the prompt asks the model to wrap each field as {value, page, bbox, confidence} for downstream citation overlays.
sample	PresetSample | null	/* serde(default) */	Optional bundled sample (input file + reference output) for preview.
fingerprint	string	/* serde(default) */	Stable sha256 fingerprint of the canonical preset file contents. Populated at registry load — not present in the on-disk JSON files. Used as a cache-invalidation token by the worker pipeline.

---

PresetSample

Pointer to a sample input + its reference output bundled with the preset.

Field	Type	Default	Description
inputPath	string	—	Path to the sample input file, relative to the preset directory.
outputPath	string	—	Path to the reference structured output, relative to the preset directory.

---

PresetSummary

Lightweight projection of Preset used by the registry list endpoint (omits the full schema and prompt to keep the payload small).

Field	Type	Default	Description
id	string	—	Preset identifier matching Preset.id.
version	string	—	Preset version matching Preset.version.
schemaName	string	—	Schema name matching Preset.schema_name.
description	string	—	One-line preset description.
category	PresetCategory	—	Top-level category.
tags	Array<string>	—	Free-form tags.
preferredCallMode	CallMode	—	Default call mode.
emitCitations	boolean	—	Whether the preset prompts the model for citations.
fingerprint	string	—	Stable fingerprint matching Preset.fingerprint.

---

ProcessingWarning

A non-fatal warning from a processing pipeline stage.

Captures errors from optional features that don’t prevent extraction but may indicate degraded results.

Field	Type	Default	Description
source	string	—	The pipeline stage or feature that produced this warning (e.g., “embedding”, “chunking”, “language_detection”, “output_format”).
message	string	—	Human-readable description of what went wrong.

---

PropertyChange

A single run-level or style-level property change.

Used for revisions that change formatting rather than text content. from and to store normalized property values when the source format exposes them; either side may be absent when the format only records one side of the change.

Field	Type	Default	Description
name	string	—	Property name, such as "bold", "italic", "font_size", or "font_color".
from	string | null	null	Value before the change, when available.
to	string | null	null	Value after the change, when available.

---

ProxyConfig

Proxy configuration for HTTP requests.

Field	Type	Default	Description
url	string	—	Proxy URL (e.g. “http://proxy:8080", “socks5://proxy:1080”).
username	string | null	null	Optional username for proxy authentication.
password	string | null	null	Optional password for proxy authentication.

---

PstMetadata

Outlook PST archive metadata.

Field	Type	Default	Description
messageCount	number	—	Total number of email messages found in the PST archive.

---

QrBoundingBox

Pixel-space bounding box of a QR code inside its source image.

Field	Type	Default	Description
x	number	—	Horizontal pixel offset of the bounding box top-left corner.
y	number	—	Vertical pixel offset of the bounding box top-left corner.
width	number	—	Width of the bounding box in pixels.
height	number	—	Height of the bounding box in pixels.

---

QrCode

One QR code decoded from an extracted image.

Field	Type	Default	Description
payload	string	—	Decoded payload (text, URL, vCard string, …).
confidence	number | null	null	Detector-reported confidence in \[0.0, 1.0\]. null when the decoder does not expose confidence (the default rqrr backend always reports Some because successful decode implies high confidence).
bbox	QrBoundingBox | null	null	Bounding box of the QR code inside the source image, in pixel coordinates (x, y of the top-left corner; width, height of the rectangle). null if the decoder did not report a bounding box.

---

RakeParams

RAKE-specific parameters.

Field	Type	Default	Description
minWordLength	number	1	Minimum word length to consider (default: 1).
maxWordsPerPhrase	number	3	Maximum words in a keyword phrase (default: 3).

Methods

default()

Signature:

static default(): RakeParams

Example:

const result = RakeParams.default();

Returns: RakeParams

---

RecognizedTable

Pre-computed table markdown for a table detection region.

Produced by the TATR-based table structure recognizer and surfaced as part of layout-aware OCR results. The struct lives here (under layout-types, pure-Rust) so that consumers who do not enable layout-detection (ORT) can still reference the type in their own code.

Field	Type	Default	Description
detectionBbox	BBox	—	Detection bbox that this table corresponds to (for matching).
cells	Array<Array<string>>	—	Table cells as a 2D vector (rows × columns).
markdown	string	—	Rendered markdown table.

---

RedactionConfig

Since: v1.0

Configuration for the redaction post-processor.

Field	Type	Default	Description
categories	Array<PiiCategory>	\[\]	Categories to redact. Empty means “every category supported by the engine.”
strategy	RedactionStrategy	RedactionStrategy.Mask	Strategy applied to every match.
ner	NerConfig | null	null	Optional NER backend — required to redact PERSON / ORGANIZATION / LOCATION categories (the pure-Rust pattern engine only covers regex-detectable PII).
preserveOffsets	boolean	true	When true, chunk byte ranges are kept consistent with the rewritten content by adjusting byte_start / byte_end after replacement. When false, chunk byte ranges still refer to the original content offsets — useful when downstream consumers want to map findings back to the original document.
customTerms	Array<RedactionTerm>	\[\]	Arbitrary user-supplied literal terms to redact. Each term is treated as a regex hit against the document, surfacing as PiiCategory.Custom(label) in RedactionFinding where label is the per-term label (defaulting to the literal value itself). Case-insensitive by default; set RedactionTerm.case_sensitive for exact match. Use this when you need to redact tenant-specific tokens (employee IDs, project codes, internal product names) without writing a custom plugin.
customPatterns	Array<RedactionPattern>	\[\]	Arbitrary user-supplied regex patterns to redact. Same surfacing semantics as custom_terms: each hit becomes a PiiCategory.Custom(label) finding. Patterns are validated at config-construction time via RedactionConfig.validate.

Methods

default()

Signature:

static default(): RedactionConfig

Example:

const result = RedactionConfig.default();

Returns: RedactionConfig

validate()

Validate user-supplied terms and patterns at config-construction time.

Compiles every RedactionPattern.pattern (with the case-insensitive inline flag where applicable) and returns the first compilation error so the caller can reject the config before the redaction pipeline runs. Pure terms (regex-escaped) cannot fail to compile, but the function still rejects empty values to avoid degenerate zero-length matches.

Signature:

validate(): void

Example:

instance.validate();

Returns: No return value.

Errors: Throws Error with a descriptive message.

---

RedactionFinding

One redaction event: which span was rewritten, why, and with what.

Field	Type	Default	Description
start	number	—	Byte-offset start in the original (pre-redaction) ExtractedDocument.content.
end	number	—	Byte-offset end (exclusive) in the original ExtractedDocument.content.
category	PiiCategory	—	PII category that fired this redaction.
strategy	RedactionStrategy	—	Strategy applied to this finding (mask, hash, token-replace, drop).
replacementToken	string	—	String that replaced the original mention. Always present; for Drop the replacement is the empty string.

---

RedactionPattern

One user-supplied regex pattern to redact.

The pattern is compiled with the Rust regex crate (no look-around). Case sensitivity is encoded in the pattern via the (?i) inline flag when Self.case_sensitive is false.

Field	Type	Default	Description
label	string	—	Custom category label surfaced in RedactionFinding.category.
pattern	string	—	Regex pattern (Rust regex crate dialect — no look-around).
caseSensitive	boolean	serde(default = "default_case_sensitive")	When true, match case-sensitively; otherwise prepend (?i) to the regex.

Methods

labeled()

Build a pattern with the given label (case-insensitive by default).

Signature:

static labeled(label: string, pattern: string): RedactionPattern

Example:

const result = RedactionPattern.labeled("value", "value");

Parameters:

Name	Type	Required	Description
label	string	Yes	The label
pattern	string	Yes	The pattern

Returns: RedactionPattern

---

RedactionReport

Audit report describing what the redaction processor found and how it replaced it.

The redactor returns this alongside the rewritten content so compliance, replay, and audit-log consumers can see exactly what fired. Offsets are relative to the original pre-redaction content and are intended for audit reconstruction only — the original bytes are dropped at the end of the pipeline.

Field	Type	Default	Description
findings	Array<RedactionFinding>	—	Individual redaction findings in original-source byte order.
totalRedacted	number	—	Total number of redactions applied across the document.

---

RedactionTerm

One user-supplied literal term to redact.

Matched as a regex-escaped substring (so callers do not need to escape metacharacters themselves). Case-insensitive by default — set Self.case_sensitive to true for exact byte-match semantics.

Field	Type	Default	Description
label	string	—	Custom category label surfaced in RedactionFinding.category.
value	string	—	Literal value to match. Regex metacharacters are escaped automatically.
caseSensitive	boolean	serde(default = "default_case_sensitive")	When true, match the value as-is; otherwise match ASCII-case-insensitively.

Methods

literal()

Build a term whose label is the literal value itself (case-insensitive).

Signature:

static literal(value: string): RedactionTerm

Example:

const result = RedactionTerm.literal("value");

Parameters:

Name	Type	Required	Description
value	string	Yes	The value

Returns: RedactionTerm

labeled()

Build a term with a custom label.

Signature:

static labeled(label: string, value: string): RedactionTerm

Example:

const result = RedactionTerm.labeled("value", "value");

Parameters:

Name	Type	Required	Description
label	string	Yes	The label
value	string	Yes	The value

Returns: RedactionTerm

---

Registry

Sorted map of preset id → Preset.

Methods

loadEmbedded()

Build the registry from preset files embedded at compile time under src/presets/library/. Validates every file against the meta-schema.

Signature:

static loadEmbedded(): Registry

Example:

const result = Registry.loadEmbedded();

Returns: Registry

Errors: Throws Error with a descriptive message.

global()

Return the global registry, loading it on first access.

Panics:

Panics if any embedded preset is malformed. The build-time validation test ensures this cannot happen for the embedded presets; a panic here indicates a build artifact problem, not a runtime error.

Signature:

static global(): Registry

Example:

const result = Registry.global();

Returns: Registry

get()

Look up a preset by its identifier.

Signature:

get(id: string): Preset | null

Example:

const result = instance.get("value");

Parameters:

Name	Type	Required	Description
id	string	Yes	The id

Returns: Preset | null

summaries()

Materialize a PresetSummary list for the public registry endpoint.

Signature:

summaries(): Array<PresetSummary>

Example:

const result = instance.summaries();

Returns: Array<PresetSummary>

len()

Number of presets currently loaded.

Signature:

len(): number

Example:

const result = instance.len();

Returns: number

isEmpty()

Whether the registry contains zero presets.

Signature:

isEmpty(): boolean

Example:

const result = instance.isEmpty();

Returns: boolean

sampleBytes()

Read raw sample bytes for <preset_id> from library/<id>/samples/<name>. Returns null when the file is absent.

Signature:

sampleBytes(presetId: string, name: string): Buffer | null

Example:

const result = instance.sampleBytes("value", "value");

Parameters:

Name	Type	Required	Description
presetId	string	Yes	The preset id
name	string	Yes	The name

Returns: Buffer | null

extendFromDir()

Load additional preset files from a runtime directory and insert them into this registry.

Reads every *.json file directly under dir (non-recursive), validates each against the meta-schema, and inserts it. Files that fail validation are rejected — the error is returned immediately and the registry is left in a partially-updated state. Existing entries with the same id are overwritten.

Returns the number of presets successfully loaded from dir.

Use case

This is the injection point for downstream catalogs that add curated presets on top of the single embedded OSS preset.

Signature:

extendFromDir(dir: string): number

Example:

const result = instance.extendFromDir("value");

Parameters:

Name	Type	Required	Description
dir	string	Yes	The dir

Returns: number

Errors: Throws Error with a descriptive message.

---

Renderer

Trait for document renderers that convert extraction results to output strings.

Renderers are typically stateless converters that transform extracted content into a specific output format (Markdown, HTML, Djot, plain text, etc.). They participate in the standard Plugin lifecycle so custom renderers can be registered from any supported binding language.

The format name is exposed via Plugin.name. For stateless renderers the Plugin lifecycle methods (version, initialize, shutdown) all take no-op defaults and need not be overridden.

Thread Safety

Renderers must be Send + Sync (inherited from Plugin).

Methods

renderResult()

Binding-safe rendering entry point for foreign-language plugin bridges.

Accepts one public extraction result and returns the rendered output.

Signature:

renderResult(result: ExtractedDocument): string

Example:

const result = instance.renderResult(new ExtractedDocument());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document

Returns: string

Errors: Throws Error with a descriptive message.

---

RerankedDocument

A single document returned by the reranker, with its position in the input and score.

index maps back to the caller’s original document list, so metadata arrays (e.g. IDs, paths) can be reordered without passing them through the reranker.

Since v5.0.

Field	Type	Default	Description
index	number	—	Position of this document in the original input documents slice.
score	number	—	Relevance score in \[0, 1\]. Higher means more relevant to the query.
document	string	—	The document text.

---

RerankerBackend

Trait for in-process reranker backend plugins.

Cross-encoders score (query, document) pairs jointly and return a raw logit per document. The dispatcher in rerank applies sigmoid to convert logits to [0, 1] scores, sorts descending by score, and truncates to top_k.

Async to match the convention used by EmbeddingBackend and other plugin traits. Host-language bridges wrap their synchronous host callables in spawn_blocking or the equivalent.

Thread safety

Backends must be Send + Sync + 'static. They are stored in Arc<dyn RerankerBackend> and may be called concurrently from xberg’s dispatcher. If the backend’s underlying model is not thread-safe, the backend itself must serialize access internally (e.g. via Mutex<Inner>).

Contract

• rerank(query, documents) MUST return exactly documents.len() scores. The dispatcher validates this before sorting and returning to callers; a non-conforming backend surfaces as a XbergError.Validation, not a panic.

• Scores are raw logits in any range — callers must NOT assume [0, 1]. The dispatcher applies sigmoid before sorting.

• rerank may be called from any thread. Its future must be Send (enforced by async_trait when #[async_trait] is used on non-WASM targets).

• shutdown() (inherited from Plugin) may be invoked concurrently with an in-flight rerank() call. Implementations must tolerate this — letting in-flight calls finish via the Arc reference and only releasing shared state that isn’t needed by rerank.

Runtime

The synchronous rerank entry uses tokio.task.block_in_place to await the trait’s async rerank, which requires a multi-thread tokio runtime. Callers running inside a current_thread runtime must use rerank_async instead.

Since v5.0.

Methods

rerank()

Score a list of documents against a query.

Returns one raw logit per document in the same order as the input. The dispatcher applies sigmoid to convert to [0, 1] scores.

Errors:

Implementations should return Plugin for backend-specific failures. The dispatcher validates the returned length against documents.len() before sorting.

Signature:

rerank(query: string, documents: Array<string>): Promise<Array<number>>

Example:

const result = await instance.rerank("value", []);

Parameters:

Name	Type	Required	Description
query	string	Yes	The query
documents	Array<string>	Yes	The documents

Returns: Array<number>

Errors: Throws Error with a descriptive message.

---

RerankerConfig

Configuration for the reranking pipeline.

Controls which model to use, how many results to return, and download/cache behavior for local ONNX models.

Since v5.0.

Field	Type	Default	Description
model	RerankerModelType	RerankerModelType.Preset	The reranker model to use (defaults to “balanced” preset if not specified).
topK	number | null	null	Return at most this many documents. null returns all. Applied after sorting by score, so the highest-scoring documents are kept.
batchSize	number	32	Batch size for local ONNX cross-encoder inference.
showDownloadProgress	boolean	false	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at info level on the xberg.model_download target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by RerankerModelType.Llm and RerankerModelType.Plugin, which download no model.
cacheDir	string | null	null	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
acceleration	AccelerationConfig | null	null	Hardware acceleration for the reranker ONNX model. Controls which execution provider (CPU, CUDA, CoreML, TensorRT) is used for local inference. Defaults to null (auto-select per platform).
maxRerankDurationSecs	number | null	60	Maximum wall-clock duration (in seconds) for a single rerank() call when using RerankerModelType.Plugin. Applies only to the in-process plugin path — protects against hung host-language backends. On timeout, the dispatcher returns Plugin instead of blocking forever. null disables the timeout. The default (60 seconds) is conservative for common in-process inference; increase for large document sets on slow hardware.

Methods

default()

Signature:

static default(): RerankerConfig

Example:

const result = RerankerConfig.default();

Returns: RerankerConfig

---

ResolvedPreset

A preset merged with caller-supplied overrides (custom schema, prompt suffix, context map). Output is what the pipeline orchestrator consumes.

Field	Type	Default	Description
id	string	—	Source preset identifier.
version	string	—	Source preset version.
fingerprint	string	—	Fingerprint of the source preset file, used as a cache token.
schemaName	string	—	Schema name forwarded to the LLM.
schema	unknown	—	Effective JSON Schema (caller override or the preset’s own).
systemPrompt	string	—	System prompt with rendered context appended.
mergeMode	MergeMode	—	Merge strategy for paginated outputs.
preferredCallMode	CallMode	—	Preferred call mode.
emitCitations	boolean	—	Whether the prompt asks for per-field citations.

---

RevisionDelta

The content changes that make up a single revision.

For insertions and deletions the content field carries the added/removed lines as DiffLine.Added / DiffLine.Removed entries. For format changes, property_changes carries normalized before/after formatting values when the source document exposes them.

Field	Type	Default	Description
content	Array<DiffLine>	\[\]	Line-level content changes for this revision.
tableChanges	Array<CellChange>	\[\]	Cell-level table changes for this revision.
propertyChanges	Array<PropertyChange>	\[\]	Formatting or metadata property changes for this revision.

---

SecurityLimits

Configuration for security limits across extractors.

All limits are intentionally conservative to prevent DoS attacks while still supporting legitimate documents.

Field	Type	Default	Description
maxArchiveSize	number	524288000	Maximum uncompressed size for archives (500 MB)
maxCompressionRatio	number	100	Maximum compression ratio before flagging as potential bomb (100:1)
maxFilesInArchive	number	10000	Maximum number of files in archive (10,000)
maxNestingDepth	number	1024	Maximum nesting depth for structures (100)
maxEntityLength	number	1048576	Maximum length of any single XML entity / attribute / token (1 MiB). This is a per-token cap, NOT a total cap — billion-laughs class attacks where a single entity expands to hundreds of MB are caught here, while normal long text content (a paragraph, a CDATA block) is caught by max_content_size instead.
maxContentSize	number	104857600	Maximum string growth per document (100 MB)
maxIterations	number	10000000	Maximum iterations per operation
maxXmlDepth	number	1024	Maximum XML depth (100 levels)
maxTableCells	number	100000	Maximum cells per table (100,000)

Methods

default()

Signature:

static default(): SecurityLimits

Example:

const result = SecurityLimits.default();

Returns: SecurityLimits

---

ServerConfig

API server configuration.

This struct holds all configuration options for the Xberg API server, including host/port settings, CORS configuration, and upload limits.

Defaults

• host: “127.0.0.1” (localhost only)
• port: 8000
• cors_origins: empty listtor (allows all origins)
• max_request_body_bytes: 104_857_600 (100 MB)
• max_multipart_field_bytes: 104_857_600 (100 MB)

Field	Type	Default	Description
host	string	—	Server host address (e.g., “127.0.0.1”, “0.0.0.0”)
port	number	—	Server port number
corsOrigins	Array<string>	\[\]	CORS allowed origins. Empty vector means allow all origins. If this is an empty listtor, the server will accept requests from any origin. If populated with specific origins (e.g., "<https://example.com">), only those origins will be allowed.
maxRequestBodyBytes	number	—	Maximum size of request body in bytes (default: 100 MB)
maxMultipartFieldBytes	number	—	Maximum size of multipart fields in bytes (default: 100 MB)

Methods

default()

Signature:

static default(): ServerConfig

Example:

const result = ServerConfig.default();

Returns: ServerConfig

listenAddr()

Get the server listen address (host:port).

Signature:

listenAddr(): string

Example:

const result = instance.listenAddr();

Returns: string

corsAllowsAll()

Check if CORS allows all origins.

Returns true if the cors_origins vector is empty, meaning all origins are allowed. Returns false if specific origins are configured.

Signature:

corsAllowsAll(): boolean

Example:

const result = instance.corsAllowsAll();

Returns: boolean

isOriginAllowed()

Check if a given origin is allowed by CORS configuration.

Returns true if:

• CORS allows all origins (empty origins list), or
• The given origin is in the allowed origins list

Signature:

isOriginAllowed(origin: string): boolean

Example:

const result = instance.isOriginAllowed("value");

Parameters:

Name	Type	Required	Description
origin	string	Yes	The origin to check (e.g., “https://example.com”)

Returns: boolean

maxRequestBodyMb()

Get maximum request body size in megabytes (rounded up).

Signature:

maxRequestBodyMb(): number

Example:

const result = instance.maxRequestBodyMb();

Returns: number

maxMultipartFieldMb()

Get maximum multipart field size in megabytes (rounded up).

Signature:

maxMultipartFieldMb(): number

Example:

const result = instance.maxMultipartFieldMb();

Returns: number

---

SitemapUrl

A URL entry from a sitemap.

Field	Type	Default	Description
url	string	—	The URL.
lastmod	string | null	null	The last modification date, if present.
changefreq	string | null	null	The change frequency, if present.
priority	string | null	null	The priority, if present.

---

SparseEmbedding

A sparse learned embedding: vocabulary term indices and their weights.

indices are ascending vocabulary token ids; values[i] is the weight for indices[i]. The two arrays always have equal length. Only strictly-positive terms are retained, so the representation is genuinely sparse.

Since v5.0.

Field	Type	Default	Description
indices	Array<number>	—	Vocabulary token ids with non-zero weight, ascending.
values	Array<number>	—	Weights parallel to SparseEmbedding.indices.

---

SparseEmbeddingConfig

Configuration for the sparse-embedding pipeline.

Controls which model to use, batching, and download/cache behavior for the local ONNX SPLADE model.

Since v5.0.

Field	Type	Default	Description
model	SparseEmbeddingModelType	SparseEmbeddingModelType.Preset	The sparse-embedding model to use (defaults to the “opensearch-v3-distill” preset).
batchSize	number	—	Batch size for local ONNX inference. SPLADE emits a \[seq, vocab\] logit tensor per document, so memory scales with batch size — keep this modest.
maxLength	number	—	Maximum token sequence length for the tokenizer.
showDownloadProgress	boolean	false	Show model download progress (local ONNX path only). When enabled, transfer progress for the model, tokenizer and config files is reported at info level on the xberg.model_download target while they download (#279). A warm Hugging Face cache transfers nothing and so reports nothing. Ignored by SparseEmbeddingModelType.Plugin, which downloads no model.
cacheDir	string | null	null	Optional alternate Hugging Face cache root for model files. When unset, hf-hub follows the standard Hugging Face environment and platform cache conventions.
acceleration	AccelerationConfig | null	null	Hardware acceleration for the sparse-embedding ONNX model.
maxEmbedDurationSecs	number | null	null	Maximum wall-clock duration (in seconds) for a single embed call when using SparseEmbeddingModelType.Plugin. null disables the timeout.

Methods

default()

Signature:

static default(): SparseEmbeddingConfig

Example:

const result = SparseEmbeddingConfig.default();

Returns: SparseEmbeddingConfig

---

SparseEmbeddingPreset

Static metadata for a bundled SPLADE preset (WASM/Android-safe, no ORT).

Since v5.0.

Field	Type	Default	Description
name	string	—	Stable preset name referenced from config.
modelRepo	string	—	HuggingFace repository hosting the ONNX model.
modelFile	string	—	Path to the ONNX file within the repo.
additionalFiles	Array<string>	—	Sibling files that must be downloaded alongside model_file.
maxLength	number	—	Maximum token sequence length.
description	string	—	Human-readable description.

---

SsrfPolicy

SSRF policy configuration.

Field	Type	Default	Description
denyPrivate	boolean	true	If true, reject URLs that resolve to private/metadata IP ranges.
maxRedirects	number	5	Maximum number of HTTP redirects to follow during validation.

---

StructuredData

Structured data (Schema.org, microdata, RDFa) block.

Field	Type	Default	Description
dataType	StructuredDataType	—	Type of structured data
rawJson	string	—	Raw JSON string representation
schemaType	string | null	null	Schema type if detectable (e.g., “Article”, “Event”, “Product”)

---

StructuredDataResult

Result of parsing a structured data file (JSON, JSONL, YAML, or TOML).

Field	Type	Default	Description
content	string	—	The extracted text content, formatted for readability.
format	string	—	The source format identifier (e.g. "json", "yaml", "toml").
metadata	Record<string, string>	—	Key-value metadata extracted from recognized text fields.
textFields	Array<string>	—	JSON paths of fields that were classified as text-bearing.
value	unknown | null	null	The parsed document as a canonical serde_json.Value tree, when the source format could be represented as one. null only for TOML inputs whose toml.Value fails to round-trip through serde_json.Value (xberg-io/xberg#155): the extractor falls back to a raw code block in that case.
flattened	Array<string>	—	Flattened path: value renderings for every leaf field, in traversal order. Previously computed and discarded (xberg-io/xberg#166); now surfaced so callers get a full-text view even when the structured renderer only emits headings/lists for a subset of fields.

---

StructuredExtractionConfig

Configuration for LLM-based structured data extraction.

Sends extracted document content to a VLM with a JSON schema, returning structured data that conforms to the schema.

Field	Type	Default	Description
schema	unknown	—	JSON Schema defining the desired output structure.
schemaName	string	serde(default = "default_schema_name")	Schema name passed to the LLM’s structured output mode.
schemaDescription	string | null	/* serde(default) */	Optional schema description for the LLM.
strict	boolean	/* serde(default) */	Enable strict mode — output must exactly match the schema.
prompt	string | null	/* serde(default) */	Custom Jinja2 extraction prompt template. When null, a default template is used. Available template variables: - {{ content }} — The extracted document text. - {{ schema }} — The JSON schema as a formatted string. - {{ schema_name }} — The schema name. - {{ schema_description }} — The schema description (may be empty).
llm	LlmConfig	—	LLM configuration for the extraction.

---

SummarizationConfig

Since: v1.0

Configuration for the summarisation post-processor.

Field	Type	Default	Description
strategy	SummaryStrategy	SummaryStrategy.Extractive	Summarisation strategy.
maxTokens	number | null	null	Maximum summary length in tokens. null lets the backend pick a default.
llm	LlmConfig | null	null	LLM configuration for the abstractive backend. Ignored when strategy = Extractive. Required when strategy = Abstractive.

---

SupportedFormat

A supported document format entry.

Represents a file extension and its corresponding MIME type that Xberg can process.

Field	Type	Default	Description
extension	string	—	File extension (without leading dot), e.g., “pdf”, “docx”
mimeType	string	—	MIME type string, e.g., “application/pdf”

---

SvgOptions

SVG-specific configuration for the image-encode pipeline.

Applies when the source image is SVG or when the output format is set to ImageOutputFormat.Svg. Available when the svg feature is active.

Used via ImageExtractionConfig.svg.

Field	Type	Default	Description
sanitize	boolean	true	Run SVG bytes through usvg sanitization (strips external href attributes, JavaScript event handlers, and foreignObject elements) even when the output format is Native. Defaults to true.
renderDpi	number	96	Target DPI when rasterizing SVG to a pixel-based format (PNG, JPEG, WebP, HEIF). The tree’s viewBox is scaled by render_dpi / 96.0 before the pixel buffer is allocated. Defaults to 96.0 (1× CSS pixel density).

Methods

default()

Signature:

static default(): SvgOptions

Example:

const result = SvgOptions.default();

Returns: SvgOptions

---

Table

Extracted table structure.

Represents a table detected and extracted from a document (PDF, image, etc.). Tables are converted to both structured cell data and Markdown format.

Field	Type	Default	Description
cells	Array<Array<string>>	\[\]	Table cells as a 2D vector (rows × columns)
markdown	string	—	Markdown representation of the table
pageNumber	number	—	Page number where the table was found (1-indexed)
boundingBox	BoundingBox | null	null	Bounding box of the table on the page (PDF coordinates: x0=left, y0=bottom, x1=right, y1=top). Only populated for PDF-extracted tables when position data is available.
tableId	string | null	null	Stable identifier shared by every tables\[\] entry that represents a fragment of the same physical table. Assigned deterministically by the extraction pipeline (e.g. a sequential "table-N" in document order); never derived from randomness or wall-clock time, so the same input document always produces the same ids. Consumers can use it to reconcile the markdown blocks in content / pages\[\].content / chunks\[\].content with the structured entries in tables\[\]. null when the extractor did not assign one. Today, same-page fragments of one physical table are already merged into a single tables\[\] entry before ids are assigned (see PDF table stitching), so in practice table_id is unique per entry rather than shared across several. A table split across a page boundary is intentionally not linked — its per-page pieces get separate ids. Sharing one id across page-boundary fragments is a known possible future extension, not implemented yet.
columns	Array<string> | null	\[\]	Header cells for this fragment, i.e. the first row of cells. Populated even when this fragment’s own header row was merged away or physically lives in a sibling fragment (see table_id), so a single fragment is interpretable in isolation. null when no header row could be determined.

---

TableCell

Individual table cell with content and optional styling.

Future extension point for rich table support with cell-level metadata.

Field	Type	Default	Description
content	string	—	Cell content as text
rowSpan	number	—	Row span (number of rows this cell spans)
colSpan	number	—	Column span (number of columns this cell spans)
isHeader	boolean	—	Whether this is a header cell

---

TableDiff

Cell-level changes for a pair of tables that share the same index.

Field	Type	Default	Description
fromIndex	number	—	Zero-based index of the table in both a.tables and b.tables.
toIndex	number	—	Zero-based index in b.tables (equal to from_index for same-dimension tables).
cellChanges	Array<CellChange>	—	Cell-level changes within the table.

---

TableGrid

Structured table grid with cell-level metadata.

Stores row/column dimensions and a flat list of cells with position info.

Field	Type	Default	Description
rows	number	—	Number of rows in the table.
cols	number	—	Number of columns in the table.
cells	Array<GridCell>	\[\]	All cells in row-major order.

---

TesseractConfig

Tesseract OCR configuration.

Provides fine-grained control over Tesseract OCR engine parameters. Most users can use the defaults, but these settings allow optimization for specific document types (invoices, handwriting, etc.).

Field	Type	Default	Description
language	Array<string>	\[\]	Language code(s) for OCR recognition. For Tesseract, languages are joined with “+”. A list is the canonical form and the only form accepted by the binding object APIs (Python, Node, PHP, WASM, etc.): \["eng", "deu"\]. When deserializing from a config file, JSON body, or the REST/MCP API, a single string is also accepted, either as one code (“eng”) or “+”-joined (“eng+deu”).
psm	number	3	Page Segmentation Mode (0-13). Common values: - 3: Fully automatic page segmentation (native default) - 6: Assume a single uniform block of text (WASM default — avoids layout-analysis hang) - 11: Sparse text with no particular order
outputFormat	string	"markdown"	Output format (“text” or “markdown”)
oem	number	3	OCR Engine Mode (0-3). - 0: Legacy engine only - 1: Neural nets (LSTM) only (usually best) - 2: Legacy + LSTM - 3: Default (based on what’s available)
minConfidence	number	0	Minimum confidence threshold (0.0-100.0). Words with confidence below this threshold may be rejected or flagged.
preprocessing	ImagePreprocessingConfig | null	null	Image preprocessing configuration. Controls how images are preprocessed before OCR. Can significantly improve quality for scanned documents or low-quality images.
enableTableDetection	boolean	true	Enable automatic table detection and reconstruction
tableMinConfidence	number	0	Minimum confidence threshold for table detection (0.0-1.0)
tableColumnThreshold	number	50	Column threshold for table detection (pixels)
tableRowThresholdRatio	number	0.5	Row threshold ratio for table detection (0.0-1.0)
useCache	boolean	true	Enable OCR result caching
classifyUsePreAdaptedTemplates	boolean	true	Use pre-adapted templates for character classification
languageModelNgramOn	boolean	false	Enable N-gram language model
tesseditDontBlkrejGoodWds	boolean	true	Don’t reject good words during block-level processing
tesseditDontRowrejGoodWds	boolean	true	Don’t reject good words during row-level processing
tesseditEnableDictCorrection	boolean	true	Enable dictionary correction
tesseditCharWhitelist	string	""	Whitelist of allowed characters (empty = all allowed)
tesseditCharBlacklist	string	""	Blacklist of forbidden characters (empty = none forbidden)
tesseditUsePrimaryParamsModel	boolean	true	Use primary language params model
textordSpaceSizeIsVariable	boolean	true	Variable-width space detection
thresholdingMethod	boolean	false	Use adaptive thresholding method

Methods

default()

Signature:

static default(): TesseractConfig

Example:

const result = TesseractConfig.default();

Returns: TesseractConfig

---

TextAnnotation

Inline text annotation — byte-range based formatting and links.

Annotations reference byte offsets into the node’s text content, enabling precise identification of formatted regions.

Field	Type	Default	Description
start	number	—	Start byte offset in the node’s text content (inclusive).
end	number	—	End byte offset in the node’s text content (exclusive).
kind	AnnotationKind	—	Annotation type.

---

TextExtractionResult

Plain text and Markdown extraction result.

Contains the extracted text along with statistics and, for Markdown files, structural elements like headers and links.

Field	Type	Default	Description
content	string	—	Extracted text content
lineCount	number	—	Number of lines
wordCount	number	—	Number of words
characterCount	number	—	Number of characters
headers	Array<string> | null	null	Markdown headers (text only, Markdown files only)

---

TextMetadata

Text/Markdown metadata.

Extracted from plain text and Markdown files. Includes word counts and, for Markdown, structural elements like headers and links.

Field	Type	Default	Description
lineCount	number	—	Number of lines in the document
wordCount	number	—	Number of words
characterCount	number	—	Number of characters
headers	Array<string> | null	\[\]	Markdown headers (headings text only, for Markdown files)

---

TokenCounter

Per-category running counter for RedactionStrategy.TokenReplace.

Methods

new()

Create a fresh counter with no previous state.

Signature:

static new(): TokenCounter

Example:

const result = TokenCounter.new();

Returns: TokenCounter

---

TokenReductionConfig

Configuration for the token-reduction pipeline.

Field	Type	Default	Description
level	ReductionLevel	ReductionLevel.Moderate	Reduction intensity level.
languageHint	string | null	null	ISO 639-1 language code hint for stopword selection (e.g. "en", "de").
preserveMarkdown	boolean	false	Preserve Markdown formatting tokens during reduction.
preserveCode	boolean	true	Preserve code block contents unchanged.
semanticThreshold	number	0.3	Cosine similarity threshold below which sentences are considered dissimilar.
enableParallel	boolean	true	Use Rayon parallel iterators for multi-core processing.
useSimd	boolean	true	Use SIMD-optimized text scanning where available.
customStopwords	Record<string, Array<string>> | null	null	Per-language custom stopword lists (language_code → stopword_list).
preservePatterns	Array<string>	\[\]	Regex patterns whose matched text is always preserved unchanged.
targetReduction	number | null	null	Target fraction of text to retain (0.0–1.0); null = no fixed target.
enableSemanticClustering	boolean	false	Group semantically similar sentences and emit only one per cluster.
preserveImportantWords	boolean	true	Skip removal of words with “important” characteristics (all-caps acronyms, words containing digits, mixed-case identifiers, very long words) during the Aggressive/Maximum common-word removal pass. true (the default) protects those words even when they would otherwise be dropped as low-value filler; false lets the frequency/ length heuristics apply uniformly to every word, including ones that look like acronyms or technical terms (#269).

Methods

default()

Signature:

static default(): TokenReductionConfig

Example:

const result = TokenReductionConfig.default();

Returns: TokenReductionConfig

---

TokenReductionOptions

Token reduction configuration.

Field	Type	Default	Description
mode	string	—	Reduction mode: “off”, “light”, “moderate”, “aggressive”, “maximum”
preserveImportantWords	boolean	true	Preserve important words (capitalized, technical terms)

Methods

default()

Signature:

static default(): TokenReductionOptions

Example:

const result = TokenReductionOptions.default();

Returns: TokenReductionOptions

---

TokenizerBackend

Trait for in-process tokenizer backend plugins.

Unlike EmbeddingBackend, this trait is synchronous: the chunk splitter calls Self.count_tokens inside its boundary search, many times per chunk, so counting must be a direct call with no async dispatch. Host-language bridges (PyO3, napi-rs, etc.) invoke their host callable synchronously on the calling thread; implementations should keep count_tokens cheap — it dominates chunking time when the backend is slow.

Lifecycle

initialize() is called once during registration, before any count_tokens call; lazy-loading implementations should load their vocabulary there. After registration succeeds, count_tokens may be called from any thread, concurrently. shutdown() runs on unregistration and may overlap an in-flight count_tokens call from a chunking run that resolved the backend earlier — implementations must tolerate this, e.g. by keeping the resources count_tokens needs alive via Arc.

Thread safety

Backends must be Send + Sync + 'static (inherited from Plugin). They are stored in Arc<dyn TokenizerBackend> and called concurrently from xberg’s chunking pipeline. If the underlying tokenizer isn’t thread-safe, the backend must serialize access internally.

Contract

• count_tokens must return a non-zero count for non-empty text. The registry probes this once at registration and rejects backends that report zero — a zero count would make every span appear to fit any budget. At runtime, a zero count for non-empty text is not trusted: the chunker substitutes the character count and logs the substitution. (An implementation may still return 0 for the empty string.)

• count_tokens must not panic; return a best-effort count for text the tokenizer can’t fully process.

• Counting should be deterministic for a given input — the splitter may evaluate overlapping spans of the same text repeatedly.

Methods

countTokens()

Count the tokens in text according to this backend’s tokenizer.

Signature:

countTokens(text: string): number

Example:

const result = instance.countTokens("value");

Parameters:

Name	Type	Required	Description
text	string	Yes	The text

Returns: number

---

TranscriptionConfig

Configuration for audio/video transcription (speech-to-text).

When present and enabled, Xberg will route audio and video files (mp3, mp4, m4a, wav, webm, etc.) through the transcription pipeline.

The heavy dependencies (ORT, hf-hub, symphonia) are only pulled when the transcription feature is enabled. The config struct itself is available under transcription-types so that ExtractionConfig round-trips on all targets.

All fields have sensible defaults. The recommended starting point is:

[extraction.transcription]
enabled = true
model = "tiny"

Field	Type	Default	Description
enabled	boolean	true	Master switch. When false, the transcription pipeline is not run. The extractor is registered for audio/video MIME types whenever the transcription feature is compiled in, independently of this flag, so an audio/video input with enabled = false fails with an XbergError.Transcription explaining how to turn transcription on — it does not fall through to another extractor.
model	WhisperModel	WhisperModel.Tiny	Whisper model size to use. Smaller = faster + lower memory. tiny is the pragmatic default for first-time users and CI.
language	string | null	null	Optional language hint (ISO-639-1 code, e.g. “en”, “de”). When null (default), the current engine falls back to English. For deterministic production output, always set this explicitly.
timestamps	boolean	false	Whether to request segment-level timestamps. When true, the decoder prompt omits <|notimestamps|> so the model emits <|x.xx|> tokens, and each transcript segment becomes its own paragraph element carrying start_ms / end_ms attributes. When false (default), all segment text is joined into a single flat paragraph with no timing attributes.
maxDurationMs	number | null	null	Hard safety limit on input duration (milliseconds). Files longer than this are rejected after decode, before model work. Default: 30 minutes. Set to null to disable (not recommended for untrusted input).
maxBytes	number | null	null	Hard safety limit on input size (bytes). Default: 512 MiB. Protects against pathological or malicious uploads.
timeoutMs	number | null	null	Wall-clock timeout for the entire transcription operation (ms). Bounds audio decode, model resolution/download, and inference together. On expiry the extraction fails with an XbergError.Transcription. null disables the bound and lets the operation run unbounded (not recommended for untrusted input). Enforced on the async extraction path only; the size and duration caps (max_bytes, max_duration_ms) are checked on every path. Default: 10 minutes.
modelCacheDir	string | null	null	Optional alternate Hugging Face cache root for Whisper models. When unset, hf-hub follows HF_HUB_CACHE, HUGGINGFACE_HUB_CACHE, HF_HOME, XDG, and platform defaults. Files remain in the standard content-addressed snapshot layout and are not copied into an Xberg cache.
allowNetwork	boolean	true	Allow network access to download models from Hugging Face Hub. When false, only previously cached models may be used. Useful for air-gapped or fully offline deployments.
verifyHash	boolean	false	Request SHA256 verification of downloaded model files. Defaults to false because the resolver downloads from mutable Hugging Face refs unless callers pin and verify models out-of-band. Explicit true requests are rejected by the model resolver until pinned checksum metadata is available.

Methods

default()

Signature:

static default(): TranscriptionConfig

Example:

const result = TranscriptionConfig.default();

Returns: TranscriptionConfig

---

Translation

Translation of the extracted content.

Holds the translated rendition of ExtractedDocument.content and (when preserve_markup was requested) the translated formatted_content. Chunks are translated in place inside ExtractedDocument.chunks[*].content rather than duplicated here.

Field	Type	Default	Description
targetLang	string	—	BCP-47 language tag the translation was produced into (e.g. "de", "fr-CA").
sourceLang	string | null	null	BCP-47 source language. null when the translation backend was asked to detect.
content	string	—	Translated plain-text body. Matches the shape of ExtractedDocument.content.
formattedContent	string | null	null	Translated markup body (Markdown / HTML / etc.) when preserve_markup was enabled on the config. null otherwise.

---

TranslationConfig

Since: v1.0

Configuration for the translation post-processor.

Field	Type	Default	Description
targetLang	string	—	BCP-47 language tag for the target language (e.g. "de", "fr-CA").
sourceLang	string | null	null	Optional explicit source language. null asks the backend to auto-detect.
preserveMarkup	boolean	/* serde(default) */	Translate the formatted (Markdown/HTML) rendition alongside plain text when formatted_content is present.
llm	LlmConfig	—	LLM configuration used for translation.

---

TreeSitterConfig

Configuration for tree-sitter language pack integration.

Controls grammar download behavior and code analysis options.

Example (TOML)

[tree_sitter]
languages = ["python", "rust"]
groups = ["web"]

[tree_sitter.process]
structure = true
comments = true
docstrings = true

Field	Type	Default	Description
enabled	boolean	true	Enable code intelligence processing (default: true). When false, tree-sitter analysis is completely skipped even if the config section is present.
cacheDir	string | null	null	Custom cache directory for downloaded grammars. When null, uses the default: ~/.cache/tree-sitter-language-pack/v{version}/libs/. Consumed both by the CLI (tree-sitter download --from-config, cache warm) and by CodeExtractor at extraction time, so that a configured cache directory is honoured wherever grammars are looked up or downloaded, not only during an explicit CLI download.
languages	Array<string> | null	null	Languages to pre-download on init (e.g., \["python", "rust"\]). Consumed only by the CLI’s tree-sitter download --from-config and cache warm commands as a pre-download hint. Extraction itself does not read this field: a given source file always processes with a single, already auto-detected language, so there is nothing for a language allowlist to gate at extraction time.
groups	Array<string> | null	null	Language groups to pre-download (e.g., \["web", "systems", "scripting"\]). Consumed only by the CLI’s tree-sitter download --from-config and cache warm commands, for the same reason as languages above.
process	TreeSitterProcessConfig	—	Processing options for code analysis.

Methods

default()

Signature:

static default(): TreeSitterConfig

Example:

const result = TreeSitterConfig.default();

Returns: TreeSitterConfig

---

TreeSitterProcessConfig

Processing options for tree-sitter code analysis.

Controls which analysis features are enabled when extracting code files.

Field	Type	Default	Description
structure	boolean	true	Extract structural items (functions, classes, structs, etc.). Default: true.
imports	boolean	true	Extract import statements. Default: true.
exports	boolean	true	Extract export statements. Default: true.
comments	boolean	false	Extract comments. Default: false.
docstrings	boolean	false	Extract docstrings. Default: false.
symbols	boolean	false	Extract symbol definitions. Default: false.
diagnostics	boolean	false	Include parse diagnostics. Default: false.
dataExtraction	boolean	false	Extract a hierarchical key/value data tree from data-format files (JSON, YAML, TOML, XML, CSV, etc.). Default: false.
chunkMaxSize	number | null	null	Maximum chunk size in bytes. null disables chunking.
contentMode	CodeContentMode	CodeContentMode.Chunks	Content rendering mode for code extraction.

Methods

default()

Signature:

static default(): TreeSitterProcessConfig

Example:

const result = TreeSitterProcessConfig.default();

Returns: TreeSitterProcessConfig

---

UrlExtractionConfig

URL ingestion and crawl configuration.

Field	Type	Default	Description
mode	UrlExtractionMode	UrlExtractionMode.Auto	URL extraction mode.
crawl	CrawlConfig	—	Crawlberg crawl configuration used for HTTP(S) URL extraction.
documentUrlPattern	string | null	null	Optional regex filter for document-discovered URLs.
maxDocumentUrlsPerResult	number | null	100	Maximum URLs to follow per extraction result.
maxTotalUrls	number | null	1000	Maximum URLs followed across the whole extraction call.
allowLocalFileInputs	boolean	true	Allow bare local filesystem path inputs.
allowFileUris	boolean	true	Allow local file:// URI inputs.

Methods

default()

Signature:

static default(): UrlExtractionConfig

Example:

const result = UrlExtractionConfig.default();

Returns: UrlExtractionConfig

---

UserChunkConfig

User-provided chunk configuration.

Field	Type	Default	Description
pageRanges	Array<PageRange> | null	\[\]	User-specified page ranges (overrides automatic chunking).
pagesPerChunk	number | null	null	User-specified pages per chunk (overrides automatic calculation).
forceChunking	boolean	—	Force chunking even for small documents.
disableChunking	boolean	—	Disable chunking even for large documents.

---

Validator

Trait for validator plugins.

Validators check extraction results for quality, completeness, or correctness. Unlike post-processors, validator errors fail fast - if a validator returns an error, the extraction fails immediately.

Use Cases

• Quality Gates: Ensure extracted content meets minimum quality standards
• Compliance: Verify content meets regulatory requirements
• Content Filtering: Reject documents containing unwanted content
• Format Validation: Verify extracted content structure
• Security Checks: Scan for malicious content

Error Handling

Validator errors are fatal - they cause the extraction to fail and bubble up to the caller. Use validators for hard requirements that must be met.

For non-fatal checks, use post-processors instead.

Thread Safety

Validators must be thread-safe (Send + Sync).

Methods

validate()

Validate an extraction result.

Check the extraction result and return Ok(()) if valid, or an error if validation fails.

Returns:

• Ok(()) if validation passes
• Err(...) if validation fails (extraction will fail)

Errors:

• XbergError.Validation - Validation failed
• Any other error type appropriate for the failure

Example - Content Length Validation

Example - Quality Score Validation

Example - Security Validation

Signature:

validate(result: ExtractedDocument, config: ExtractionConfig): Promise<void>

Example:

await instance.validate(new ExtractedDocument(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extraction result to validate
config	ExtractionConfig	Yes	Extraction configuration

Returns: No return value.

Errors: Throws Error with a descriptive message.

shouldValidate()

Optional: Check if this validator should run for a given result.

Allows conditional validation based on MIME type, metadata, or content. Defaults to true (always run).

Returns:

true if the validator should run, false to skip.

Signature:

shouldValidate(result: ExtractedDocument, config: ExtractionConfig): boolean

Example:

const result = instance.shouldValidate(new ExtractedDocument(), new ExtractionConfig());

Parameters:

Name	Type	Required	Description
result	ExtractedDocument	Yes	The extracted document
config	ExtractionConfig	Yes	The extraction config

Returns: boolean

priority()

Optional: Get the validation priority.

Higher priority validators run first. Useful for ordering validation checks (e.g., run cheap validations before expensive ones).

Default priority is 50.

Returns:

Priority value (higher = runs earlier).

Signature:

priority(): number

Example:

const result = instance.priority();

Returns: number

---

XlsxAppProperties

Application properties from docProps/app.xml for XLSX

Contains Excel-specific document metadata.

Field	Type	Default	Description
application	string | null	null	Application name (e.g., “Microsoft Excel”)
appVersion	string | null	null	Application version
docSecurity	number | null	null	Document security level
scaleCrop	boolean | null	null	Scale crop flag
linksUpToDate	boolean | null	null	Links up to date flag
sharedDoc	boolean | null	null	Shared document flag
hyperlinksChanged	boolean | null	null	Hyperlinks changed flag
company	string | null	null	Company name
worksheetNames	Array<string>	\[\]	Worksheet names

---

XmlExtractionResult

XML extraction result.

Contains extracted text content from XML files along with structural statistics about the XML document.

Field	Type	Default	Description
content	string	—	Extracted text content (XML structure filtered out)
elementCount	number	—	Total number of XML elements processed
uniqueElements	Array<string>	—	List of unique element names found (sorted)

---

XmlMetadata

XML metadata extracted during XML parsing.

Provides statistics about XML document structure.

Field	Type	Default	Description
elementCount	number	—	Total number of XML elements processed
uniqueElements	Array<string>	\[\]	List of unique element tag names (sorted)

---

YakeParams

YAKE-specific parameters.

Field	Type	Default	Description
windowSize	number	2	Window size for co-occurrence analysis (default: 2). Controls the context window for computing co-occurrence statistics.

Methods

default()

Signature:

static default(): YakeParams

Example:

const result = YakeParams.default();

Returns: YakeParams

---

YearRange

Year range for bibliographic metadata.

Field	Type	Default	Description
min	number | null	null	Earliest (minimum) year in the range.
max	number | null	null	Latest (maximum) year in the range.
years	Array<number>	/* serde(default) */	All individual years present in the collection.

---

Enums

ExecutionProviderType

ONNX Runtime execution provider type.

Determines which hardware backend is used for model inference. Auto (default) selects the best available provider per platform.

Value	Description
Auto	Auto-select: CoreML on macOS, CUDA on Linux, CPU elsewhere.
Cpu	CPU execution provider (always available).
CoreMl	Apple CoreML (macOS/iOS Neural Engine + GPU).
Cuda	NVIDIA CUDA GPU acceleration.
TensorRt	NVIDIA TensorRT (optimized CUDA inference).

---

ImageOutputFormat

Target format for re-encoding extracted images.

Controls whether and how extracted images are normalised to a uniform container format before being returned in ExtractedDocument.images. The default (Native) preserves the format produced by each extractor without any additional encode pass.

Callers that need uniform output — e.g. cloud pipelines that always store WebP thumbnails — set this once on ImageExtractionConfig.output_format rather than re-encoding downstream.

Serde shape

Uses a tagged enum: {"type": "native"}, {"type": "png"}, {"type": "jpeg", "quality": 90}, etc.

Value	Description
Native	Preserve whatever format the extractor produced (default). No re-encode pass is performed. ExtractedImage.format reflects the source format: JPEG for embedded PDF images, PNG for rasterised content, or the native container format from office documents.
Png	Re-encode all extracted images as PNG (lossless).
Jpeg	Re-encode all extracted images as JPEG at the given quality level. quality must be in 1..=100. Values outside this range are clamped and a warning is emitted. Higher values produce larger files with less artefacting; 85 is a reasonable default. — Fields: quality: number
Webp	Re-encode all extracted images as WebP at the given quality level. quality must be in 1..=100. Values outside this range are clamped and a warning is emitted. 80 is a reasonable default. — Fields: quality: number
Heif	Re-encode all extracted images as HEIF/HEIC at the given quality level. quality must be in 1..=100. Values outside this range are clamped and a warning is emitted. 80 is a reasonable default. The encode path requires the heic feature; on builds without it, selecting this variant returns an EncodeFailed warning and leaves the image untouched. — Fields: quality: number
Svg	Output pure-vector SVG. Lossless. Raster sources are not re-encoded (a warning is emitted and the image bytes are left untouched). When the source is already SVG, the bytes are passed through the usvg sanitizer (strips external hrefs, JS event handlers, and foreignObject elements) when SvgOptions.sanitize is true. Requires the svg feature.

---

ExtractInputKind

Source kind for ExtractInput.

Value	Description
Bytes	Raw in-memory bytes.
Uri	A filesystem path, file:// URI, or HTTP(S) URL.

---

UrlExtractionMode

URL extraction mode.

Value	Description
Auto	Classify HTTP(S) resources after fetch.
Document	Treat the URI as a single remote document/page.
Crawl	Crawl from the seed URI and extract discovered pages/documents.

---

BreadcrumbTarget

Deprecated and inert. Chunking no longer writes a heading breadcrumb into content for either variant of this enum — see the revised design adopted in https://github.com/xberg-io/xberg/issues/1393. Setting this field has no effect on chunking output any more. It is kept only so the ~15 alef-generated binding packages that construct it keep compiling; removing it outright is a separate, coordinated breaking change.

Why this became inert

The original design (this enum, plus ChunkingConfig.prepend_heading_context) let Content mode prepend the heading breadcrumb directly into a chunk’s content. GH#1393’s follow-up discussion argued that a single flag on the chunker cannot serve all three retrieval consumers of the same chunk: dense/ embedding retrieval wants the breadcrumb inline, but lexical (BM25/TF-IDF) and sparse learned (SPLADE) retrieval are actively harmed by it — SPLADE worse than BM25, because its term-expansion pulls each heading’s whole learned neighbourhood (e.g. "Authentication" → auth, login, credential, oauth) into every chunk of that section, and that damage cannot be corrected by re-indexing since the expansion comes from a pretrained encoder, not the collection being indexed. Mutating content also desynced it from byte_start/byte_end (#1294): chunk.content.len() != byte_end - byte_start whenever a breadcrumb had been prepended, so slicing the source document by a chunk’s own offsets silently returned different text than content.

The revised design removes the mutation entirely: chunk.content now always equals the exact [byte_start, byte_end) source span, regardless of this enum’s value or prepend_heading_context.

What to do instead

Call render_heading_breadcrumb explicitly at index time, with a chunk’s (always-clean) content and its heading_context — only for the consumer that wants the breadcrumb inline (typically dense/embedding). BM25 and SPLADE consumers need no special handling: index chunk.content as returned. See the rag module docs for the full per-consumer guidance.

Value	Description
Content	Inert (#1393). Previously prepended the heading breadcrumb into chunk content; no longer has any effect — content is left untouched, exactly like Metadata. Kept as the default only for wire/API compatibility.
Metadata	Inert (#1393), and was already a no-op on content before this change. Kept only for backward compatibility, since Content is no longer distinguishable from it.

---

OutputFormat

Output format for extraction results.

Controls the format of the content field in ExtractedDocument. When set to Markdown, Djot, or Html, the output uses that format. Plain returns the raw extracted text. Structured is currently a metadata-only label: derive_extraction_result returns null for it (see extraction/derive.rs), so no renderer runs and the content is left exactly as Plain would leave it. Only metadata.output_format differs. It does NOT attach OCR element data, bounding boxes or confidence scores.

Value	Description
Plain	Plain text content only (default)
Markdown	Markdown format
Djot	Djot markup format
Html	HTML format
Json	JSON tree format with heading-driven sections.
Structured	Metadata-only label; content is identical to OutputFormat.Plain. No dedicated renderer exists yet, so this attaches no OCR element metadata. See the enum-level docs above.
DocTags	Docling DocTags format (tables rendered as OTSL).
Custom	Custom renderer registered via the RendererRegistry. The string is the renderer name (e.g., “docx”, “latex”). — Fields: 0: string

---

JupyterCellRendering

Controls how Jupyter notebook code cells are rendered during extraction.

A code cell carries both its source and any outputs that were saved in the notebook. Callers ingesting notebooks for AI agents want different slices of this depending on the task. Xberg never executes cells — Outputs and Both only surface outputs already stored in the .ipynb.

This toggle governs a code cell’s source body and its saved outputs. Markdown (prose) cells and structural markers (kernel language, cell id, tags, execution count) are unaffected — prose always renders and markers orient the reader regardless of mode.

Value	Description
Source	Render the code source as a fenced code block; omit saved outputs.
Outputs	Omit the code source; render only the saved cell outputs.
Both	Render both the code source and the saved outputs (default; preserves the historical behavior).

---

HtmlTheme

Built-in HTML theme selection.

Value	Description
Default	Sensible defaults: system font stack, neutral colours, readable line measure. CSS custom properties (--kb-*) are all defined so user CSS can override individual values.
GitHub	GitHub Markdown-inspired palette and spacing.
Dark	Dark background, light text.
Light	Minimal light theme with generous whitespace.
Unstyled	No built-in stylesheet emitted. CSS custom properties are still defined on :root so user stylesheets can reference var(--kb-*) tokens.

---

LateInteractionModelType

Late-interaction model types supported by Xberg.

Since v5.0.

Value	Description
Preset	Use a preset ColBERT model (recommended). — Fields: name: string
Custom	Use a custom ColBERT ONNX model from HuggingFace. — Fields: modelId: string, modelFile: string, additionalFiles: Array<string>, maxLength: number
Plugin	In-process late-interaction backend registered via the plugin system. — Fields: name: string

---

TableModel

Which table structure recognition model to use.

Controls the model used for table cell detection within layout-detected table regions. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
Tatr	TATR (Table Transformer) – default, 30MB, DETR-based row/column detection.
SlanetWired	SLANeXT wired variant – 365MB, optimized for bordered tables.
SlanetWireless	SLANeXT wireless variant – 365MB, optimized for borderless tables.
SlanetPlus	SLANet-plus – 7.78MB, lightweight general-purpose.
SlanetAuto	Classifier-routed SLANeXT: auto-select wired/wireless per table. Uses PP-LCNet classifier (6.78MB) + both SLANeXT variants (730MB total).
Disabled	Disable table structure model inference entirely; use heuristic path only.

---

TableOverlapPreference

How to resolve overlapping native vs layout (TATR/SLANeXT) tables.

When both native oxide detection and the layout table model produce a table for the same page region, one must be dropped. This controls which one wins. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
Content	Keep whichever table carries more content (cell count + markdown length). This is the historical default. TATR/SLANeXT tables usually recognize more cells and therefore win, which maximizes table-structure F1 but can lower text F1 when the recognized cell reflow diverges from the source reading order.
Native	Prefer the native oxide table when it overlaps a layout table. Native tables preserve the source reading order, which scores higher on text F1 for documents where the layout model’s cell reflow diverges from the ground truth.
Layout	Prefer the layout (TATR/SLANeXT) table when it overlaps a native table.

---

LayoutStrategy

Which PDF pages the layout model runs on.

Layout detection renders each selected page to a raster and runs ONNX inference on it, which dominates extraction cost. This controls page selection; LayoutStrategy.Always preserves the historical behavior of running on every page. Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
Always	Run layout detection unconditionally on every page.
Auto	Pre-screen each page with cheap geometry signals and run the model only on pages likely to benefit (multi-column, table-bearing, figure-heavy, form-like, or rotated pages). Pages the pre-screen skips are processed exactly like pages where the model ran and found no regions. On the OCR path only inference is skipped; page rasters are still produced because OCR consumes them. For non-PDF inputs Auto behaves as LayoutStrategy.Always.

---

CredentialProviderConfig

Since: v1.1

Managed credential-provider configuration for OAuth2/STS-based authentication modes liter-llm cannot express via a static api_key. See LlmConfig.credential_provider.

Debug is implemented by hand: CredentialProviderConfig.AzureAd’s client_secret is a credential and must never be printed, matching LlmConfig’s own redaction policy. The other variants carry no secret material — CredentialProviderConfig.VertexOauth2 and CredentialProviderConfig.BedrockWebIdentity reference a file path, never the key or token itself.

Value	Description
AzureAd	Azure AD OAuth2 client-credentials flow (Azure OpenAI / Azure Cognitive Services). — Fields: tenantId: string, clientId: string, clientSecret: string, scope: string
VertexOauth2	Google Vertex AI OAuth2 via a service-account JSON key file on disk. Points at a file path rather than embedding the key inline: the key file contains an RSA private key — stronger secret material than an API key — and LlmConfig must never carry that directly, matching the credential-handling policy the rest of this module follows. — Fields: serviceAccountKeyFile: string, scope: string
VertexAdc	Google Vertex AI Application Default Credentials, resolved from the GCE/GKE/Cloud Run metadata server. Carries no secret material at all. — Fields: scope: string
BedrockWebIdentity	AWS STS AssumeRoleWithWebIdentity (EKS IRSA / OIDC federation) for Bedrock. — Fields: roleArn: string, tokenFile: string, sessionName: string, region: string

---

CallMode

How a structured-extraction preset is dispatched to the model.

This is the preset-facing call mode (the preferred_call_mode field of a Preset). The structured pipeline has a richer runtime-only decision enum with skip and fallback states; this 3-variant type is the stable, serializable surface presets and bindings depend on.

Value	Description
TextOnly	Use the extracted text only.
VisionOnly	Use rasterized page images only.
TextPlusVision	Provide both extracted text and page images to the model.

---

MergeMode

How partial results from multiple model calls (e.g. per page batch) are combined.

Canonical home for the merge strategy referenced by presets and by the structured pipeline’s post-processing. There is intentionally only one merge type across the crate — do not introduce a second.

Value	Description
ObjectMerge	Deep-merge JSON objects field by field (later calls fill missing fields).
ArrayConcat	Concatenate top-level arrays across calls.
ObjectFirst	Keep the first non-empty result; ignore subsequent calls.

---

NerBackendKind

NER backend selector.

Value	Description
Onnx	xberg-gliner ONNX inference. Requires ner-onnx feature. Models download lazily from xberg-io/gliner-models.
Llm	liter-llm zero-shot NER via structured-output prompts. Requires ner-llm feature. Useful when domain-specific categories outstrip the ONNX taxonomy.

---

VlmFallbackPolicy

Policy controlling when VLM (Vision Language Model) OCR is used as a fallback.

This knob is syntactic sugar over the explicit OcrPipelineConfig stage ordering. When vlm_fallback is set and pipeline is null, an equivalent pipeline is synthesised at extraction time:

• VlmFallbackPolicy.Disabled — no synthesis; single-backend mode (default).

• VlmFallbackPolicy.OnLowQuality — tries the classical backend first; if the result scores below quality_threshold, tries VLM.

• VlmFallbackPolicy.Always — skips the classical backend and sends every page to the VLM.

When OcrConfig.pipeline is explicitly set, vlm_fallback is ignored — the explicit pipeline takes precedence.

Errors:

Both OnLowQuality and Always require OcrConfig.vlm_config to be Some. Constructing an OcrConfig with one of these policies but no vlm_config is detected by OcrConfig.validate and will surface as a Validation error at extraction time, not a panic.

Value	Description
Disabled	No VLM fallback (default). Behaves identically to the pre-policy single-backend mode.
OnLowQuality	Try the classical OCR backend first. If the quality score is below quality_threshold, send the page to the VLM. quality_threshold is in the \[0.0, 1.0\] range produced by calculate_quality_score. A value of 0.5 is a reasonable starting point; calibrate with the Stage 0 benchmark harness. — Fields: qualityThreshold: number
Always	Skip the classical OCR backend entirely. Every page is sent to the VLM.

---

OcrStrategy

Which pages of a PDF get OCR’d when neither force_ocr nor force_ocr_pages applies.

Value	Description
Auto	OCR only when the native text layer fails a quality check (default). A scanner’s invisible OCR sidecar passes that check, so scanned pages carrying one are extracted natively. Use OcrStrategy.ScannedPages to OCR them instead.
ScannedPages	Additionally OCR every page that looks like a scan. Pages are graded on raster coverage, whether the text layer is invisible or absent, the image codec, and the producer. Pages at or above min_confidence are OCR’d; the rest keep native text and still go through the Auto quality check. Detects that a text layer came from a scanner, not whether it is accurate, so a page carrying a good sidecar is OCR’d too. — Fields: minConfidence: number

---

TableChunkingMode

Controls how markdown tables are handled when they exceed the chunk size limit.

Only applies when chunker_type is Markdown.

Variants

• Split - Default behavior: tables are split at row boundaries like any other block element. Continuation chunks contain only data rows without the header, which can break downstream consumers that need column context.

• RepeatHeader - Prepend the table header (header row + separator row) to every continuation chunk that contains data rows from the same table. Adds a small amount of duplicate text but ensures each chunk is self-contained for extraction, search, and LLM consumption.

Value	Description
Split	Split tables at row boundaries (default). Continuation chunks have no header.
RepeatHeader	Prepend the table header to every chunk that continues a split table.

---

ChunkerType

Type of text chunker to use.

Variants

• Text - Generic text splitter, splits on whitespace and punctuation
• Markdown - Markdown-aware splitter, preserves formatting and structure
• Yaml - YAML-aware splitter, creates one chunk per top-level key
• Semantic - Topic-aware chunker. With an EmbeddingConfig, splits at embedding-based topic shifts tuned by topic_threshold (default 0.75, lower = more splits). Without an embedding, falls back to a structural-boundary heuristic (ALL-CAPS headers, numbered sections, blank-line paragraphs) and merges groups into chunks capped at max_characters (default 1000). topic_threshold has no effect in the fallback path. For best results, pair with an embedding model.

Value	Description
Text	Generic whitespace- and punctuation-aware text splitter (default).
Markdown	Markdown-aware splitter that preserves heading and code-block boundaries.
Yaml	YAML-aware splitter that creates one chunk per top-level key.
Semantic	Topic-aware chunker that splits at embedding-based topic shifts.

---

ChunkSizing

How chunk size is measured.

Defaults to Characters (Unicode character count). When using token-based sizing, chunks are sized by token count according to the specified tokenizer.

Token-based sizing uses HuggingFace tokenizers loaded at runtime, or a tokenizer backend you register yourself. Any tokenizer available on HuggingFace Hub can be used, including OpenAI-compatible tokenizers (e.g., Xenova/gpt-4o, Xenova/cl100k_base). To size chunks with your own tokenizer instead (llama.cpp/GGUF vocabularies, SentencePiece models, custom vocabs), register a TokenizerBackend with register_tokenizer_backend and set model to the registered name.

Value	Description
Characters	Size measured in Unicode characters (default).
Tokenizer	Size measured in tokens from a HuggingFace tokenizer or a registered tokenizer backend. — Fields: model: string, cacheDir: string

---

EmbeddingModelType

Embedding model types supported by Xberg.

Value	Description
Preset	Use a preset model configuration (recommended) — Fields: name: string
Custom	Use a custom ONNX model from HuggingFace — Fields: modelId: string, dimensions: number
Llm	Provider-hosted embedding model via liter-llm. Uses the model specified in the nested LlmConfig (e.g., "openai/text-embedding-3-small"). — Fields: llm: LlmConfig
Plugin	In-process embedding backend registered via the plugin system. The caller registers an EmbeddingBackend once (e.g. a wrapper around an already-loaded llama-cpp-python, sentence-transformers, or tuned ONNX model), then references it by name in config. Xberg calls back into the registered backend during chunking and standalone embed requests — no HuggingFace download, no ONNX Runtime requirement, no HTTP sidecar. When this variant is selected, only the following EmbeddingConfig fields apply: normalize (post-call L2 normalization) and max_embed_duration_secs (dispatcher timeout). Model-loading fields (batch_size, cache_dir, show_download_progress, acceleration) are ignored — the host owns the model lifecycle, so there is no download to report progress for. Semantic chunking falls back to ChunkingConfig.max_characters when this variant is used, since there is no preset to look a chunk-size ceiling up against — size your context window via max_characters directly. See register_embedding_backend. — Fields: name: string

---

RerankerHead

Selects how a local ONNX reranker’s raw output tensor is turned into a score.

• RerankerHead.CrossEncoder — classic single-logit cross-encoder head: the model emits [batch, 1] (or [batch]) logits; the caller applies sigmoid to get a [0, 1] score. This is the original, unchanged path.

• RerankerHead.Qwen3Generative — Qwen3 generative-reranker head: the model emits [batch, seq, vocab] logits; the score is P("yes") read from the last token’s logits over the “yes”/“no” vocabulary entries, via a softmax over those two logits. Already a [0, 1] probability — no sigmoid is applied.

Since v5.0.

Value	Description
CrossEncoder	Single-logit cross-encoder head (sigmoid applied by the caller).
Qwen3Generative	Qwen3 generative-reranker head (softmax over yes/no token logits).

---

RerankerModelType

Reranker model types supported by Xberg.

Since v5.0.

Value	Description
Preset	Use a preset cross-encoder model (recommended). — Fields: name: string
Custom	Use a custom ONNX cross-encoder from HuggingFace. — Fields: modelId: string, modelFile: string, additionalFiles: Array<string>, maxLength: number, head: RerankerHead
Llm	Provider-hosted reranker via liter-llm (e.g. Cohere, Jina, Voyage). The model in the nested LlmConfig must be a rerank-capable model ID (e.g. "cohere/rerank-english-v3.0"). — Fields: llm: LlmConfig
Plugin	In-process reranker registered via the plugin system. The caller registers a RerankerBackend once (e.g. a wrapper around a sentence-transformers cross-encoder or a provider client), then references it by name in config. Xberg calls back into the registered backend — no HuggingFace download, no ONNX Runtime requirement. When this variant is selected, only max_rerank_duration_secs applies. Model-loading fields (batch_size, cache_dir, show_download_progress, acceleration) are ignored — the host owns the model lifecycle, so there is no download to report progress for. See register_reranker_backend. — Fields: name: string

---

SparseEmbeddingModelType

Sparse-embedding model types supported by Xberg.

Since v5.0.

Value	Description
Preset	Use a preset SPLADE model (recommended). — Fields: name: string
Custom	Use a custom SPLADE (BertForMaskedLM) ONNX model from HuggingFace. — Fields: modelId: string, modelFile: string, additionalFiles: Array<string>, maxLength: number
Plugin	In-process sparse-embedding backend registered via the plugin system. — Fields: name: string

---

WhisperModel

Supported Whisper model sizes.

These map to published ONNX exports on Hugging Face (onnx-community or similar orgs). The actual filenames and repos are resolved inside the transcription engine.

Value	Description
Tiny	Smallest, fastest, lowest quality. Good default for development and CI.
Base	Reasonable quality/speed tradeoff.
Small	Better accuracy with higher memory and cache use.
Medium	High quality; slower and more memory-intensive.
LargeV3	Best quality (large-v3). Use only when latency and memory use are acceptable.

---

CodeContentMode

Content rendering mode for code extraction.

Controls how extracted code content is represented in the content field of ExtractedDocument.

Value	Description
Chunks	Use TSLP semantic chunks as content (default).
Raw	Use raw source code as content.
Structure	Emit function/class headings + docstrings (no code bodies).

---

ListType

Type of list detection.

Value	Description
Bullet	Bullet points (-, *, •, etc.)
Numbered	Numbered lists (1., 2., etc.)
Lettered	Lettered lists (a., b., A., B., etc.)
Indented	Indented items

---

OcrBackendType

OCR backend types.

Value	Description
Tesseract	Tesseract OCR (native Rust binding)
PaddleOcr	PaddleOCR (Python-based, via FFI)
Candle	Candle-based VLM OCR (TrOCR, PaddleOCR-VL).
Custom	Name-selected built-in or third-party OCR backend.

---

ProcessingStage

Processing stages for post-processors.

Post-processors are executed in stage order (Early → Middle → Late). Use stages to control the order of post-processing operations.

Value	Description
Early	Early stage - foundational processing. Use for: - Language detection - Character encoding normalization - Entity extraction (NER) - Text quality scoring
Middle	Middle stage - content transformation. Use for: - Keyword extraction - Token reduction - Text summarization - Semantic analysis
Late	Late stage - final enrichment. Use for: - Custom user hooks - Analytics/logging - Final validation - Output formatting

---

ReductionLevel

Intensity level for the token-reduction pipeline.

Value	Description
Off	No reduction applied; text is returned as-is.
Light	Remove only the most common stopwords.
Moderate	Balanced stopword removal and redundancy filtering.
Aggressive	Aggressive filtering; may remove less common content words.
Maximum	Maximum compression; prioritizes brevity over completeness.

---

PdfAnnotationType

Type of PDF annotation.

Value	Description
Text	Sticky note / text annotation
Highlight	Highlighted text region
Link	Hyperlink annotation
Stamp	Rubber stamp annotation
Underline	Underline text markup
StrikeOut	Strikeout text markup
Squiggly	Squiggly (wavy) underline text markup — Since: v1.1
Ink	Freehand drawing (ink) annotation — Since: v1.1
Square	Rectangle/box shape annotation — Since: v1.1
Circle	Ellipse/oval shape annotation — Since: v1.1
Polygon	Closed polygon shape annotation — Since: v1.1
PolyLine	Open polyline shape annotation — Since: v1.1
Line	Line annotation — Since: v1.1
Caret	Caret (text-insertion marker) annotation — Since: v1.1
FileAttachment	Embedded file attachment annotation — Since: v1.1
Sound	Embedded sound annotation — Since: v1.1
Movie	Embedded movie annotation — Since: v1.1
Other	Any other annotation type

---

BlockType

Types of block-level elements in Djot.

Value	Description
Paragraph	Standard prose paragraph.
Heading	Section heading (level stored in FormattedBlock.level).
Blockquote	Block quotation container.
CodeBlock	Fenced or indented code block.
ListItem	Individual item within a list.
OrderedList	Numbered (ordered) list container.
BulletList	Unnumbered (bullet) list container.
TaskList	Task / checkbox list container.
DefinitionList	Definition list container.
DefinitionTerm	Term part of a definition list entry.
DefinitionDescription	Description / definition part of a definition list entry.
Div	Generic div container with optional attributes.
Section	Logical section container, often associated with a heading.
ThematicBreak	Horizontal rule / thematic break.
RawBlock	Raw content block in a specified format (e.g. HTML, LaTeX).
MathDisplay	Display-mode mathematical expression.

---

InlineType

Types of inline elements in Djot.

Value	Description
Text	Plain text run.
Strong	Bold / strong emphasis.
Emphasis	Italic / regular emphasis.
Highlight	Highlighted text (marker pen).
Subscript	Subscript text.
Superscript	Superscript text.
Insert	Inserted text (tracked change).
Delete	Deleted text (tracked change).
Code	Inline code span.
Link	Hyperlink with URL.
Image	Inline image reference.
Span	Generic inline span with optional attributes.
Math	Inline mathematical expression.
RawInline	Raw inline content in a specified format.
FootnoteRef	Footnote reference marker.
Symbol	Named symbol or emoji shortcode.

---

RelationshipKind

Semantic kind of a relationship between document elements.

Value	Description
FootnoteReference	Footnote marker -> footnote definition.
CitationReference	Citation marker -> bibliography entry.
InternalLink	Internal anchor link (#id) -> target heading/element.
Caption	Caption paragraph -> figure/table it describes.
Label	Label -> labeled element (HTML <label for>, LaTeX \label{}).
TocEntry	TOC entry -> target section.
CrossReference	Cross-reference (LaTeX \ref{}, DOCX cross-reference field).

---

ContentLayer

Content layer classification for document nodes.

Replaces separate body/furniture arrays with per-node granularity.

Value	Description
Body	Main document body content.
Header	Page/section header (running header).
Footer	Page/section footer (running footer).
Footnote	Footnote content.

---

NodeContent

Tagged enum for node content. Each variant carries only type-specific data.

Uses #[serde(tag = "node_type")] to avoid “type” keyword collision in Go/Java/TypeScript bindings.

Value	Description
Title	Document title. — Fields: text: string
Heading	Section heading with level (1-6). — Fields: level: number, text: string
Paragraph	Body text paragraph. — Fields: text: string
List	List container — children are ListItem nodes. — Fields: ordered: boolean
ListItem	Individual list item. — Fields: text: string
Table	Table with structured cell grid. — Fields: grid: TableGrid
Image	Image reference. — Fields: description: string, imageIndex: number, src: string
Code	Code block. — Fields: text: string, language: string
Quote	Block quote — container, children carry the quoted content.
Formula	Mathematical formula / equation. — Fields: text: string
Footnote	Footnote reference content. — Fields: text: string
Comment	Reviewer/editor comment content (e.g. DOCX comments). Distinct from NodeContent.Footnote (xberg-io/xberg#300): comments and footnotes both reach the internal document via a marker/definition pair, but a consumer needs to tell a reviewer comment apart from an authored footnote. — Fields: text: string — Since: v1.1
Group	Logical grouping container (section, key-value area). heading_level + heading_text capture the section heading directly rather than relying on a first-child positional convention. — Fields: label: string, headingLevel: number, headingText: string
PageBreak	Page break marker.
Slide	Presentation slide container — children are the slide’s content nodes. — Fields: number: number, title: string
DefinitionList	Definition list container — children are DefinitionItem nodes.
DefinitionItem	Individual definition list entry with term and definition. — Fields: term: string, definition: string
Citation	Citation or bibliographic reference. — Fields: key: string, text: string
Admonition	Admonition / callout container (note, warning, tip, etc.). Children carry the admonition body content. — Fields: kind: string, title: string
RawBlock	Raw block preserved verbatim from the source format. Used for content that cannot be mapped to a semantic node type (e.g. JSX in MDX, raw LaTeX in markdown, embedded HTML). — Fields: format: string, content: string
MetadataBlock	Structured metadata block (email headers, YAML frontmatter, etc.).

---

AnnotationKind

Types of inline text annotations.

Value	Description
Bold	Bold (strong) text formatting.
Italic	Italic (emphasis) text formatting.
Underline	Underlined text.
Strikethrough	Strikethrough text.
Code	Inline code span.
Subscript	Subscript text.
Superscript	Superscript text.
Link	Hyperlink annotation. — Fields: url: string, title: string
Highlight	Highlighted text (PDF highlights, HTML <mark>).
Color	Text color (CSS-compatible value, e.g. “#ff0000”, “red”). — Fields: value: string
FontSize	Font size with units (e.g. “12pt”, “1.2em”, “16px”). — Fields: value: string
Custom	Extensible annotation for format-specific styling. — Fields: name: string, value: string

---

EntityCategory

Standard entity categories produced by built-in NER backends.

The Custom(String) variant lets caller-supplied categories (e.g. LLM schemas) flow through without losing fidelity to the consumer.

Value	Description
Person	A person’s name.
Organization	A company, institution, or organisation name.
Location	A geographic location (city, country, address).
Date	A calendar date.
Time	A time of day or duration.
Money	A monetary amount with optional currency.
Percent	A percentage value.
Email	An email address.
Phone	A phone number.
Url	A URL or URI.
Custom	A caller-supplied custom category label. — Fields: 0: string

---

ExtractionMethod

How the extracted text was produced.

Value	Description
Native	Text extracted directly from the document’s native format (no OCR).
Ocr	All text was obtained via OCR (e.g. scanned image-only PDF).
Mixed	Text came from a combination of native extraction and OCR.

---

ChunkType

Semantic structural classification of a text chunk.

Assigned by the heuristic classifier in chunking.classifier. Defaults to Unknown when no rule matches. Designed to be extended in future versions without breaking changes.

Value	Description
Heading	Section heading or document title.
PartyList	Party list: names, addresses, and signatories.
Definitions	Definition clause (“X means…”, “X shall mean…”).
OperativeClause	Operative clause containing legal/contractual action verbs.
SignatureBlock	Signature block with signatures, names, and dates.
Schedule	Schedule, annex, appendix, or exhibit section.
TableLike	Table-like content with aligned columns or repeated patterns.
Formula	Mathematical formula or equation.
CodeBlock	Code block or preformatted content.
Function	Function or method definition (tree-sitter structured code chunking).
Class	Class, struct, interface, or trait definition (tree-sitter structured code chunking).
Module	Module, namespace, or top-level file scope (tree-sitter structured code chunking).
Image	Embedded or referenced image content.
OrgChart	Organizational chart or hierarchy diagram.
Diagram	Diagram, figure, or visual illustration.
Unknown	Unclassified or mixed content.

---

ImageKind

Heuristic classification of what an image likely depicts.

Value	Description
Photograph	Photographic image (natural scene, photograph)
Diagram	Technical or schematic diagram
Chart	Chart, graph, or plot
Drawing	Freehand or technical drawing
TextBlock	Text-heavy image (scanned text, document)
Decoration	Decorative element or border
Logo	Logo or brand mark
Icon	Small icon
TileFragment	Fragment of a larger tiled image (tile of a technical drawing)
Mask	Mask or transparency map
PageRaster	Full-page render produced during OCR preprocessing; used as a citation thumbnail.
Unknown	Could not classify with reasonable confidence

---

ResultFormat

Result-shape selection for extraction results.

Distinct from OutputFormat (which controls rendering — Plain, Markdown, HTML, etc.). ResultFormat controls the shape of the result: a unified content blob vs. an element-based decomposition.

Value	Description
Unified	Unified format with all content in content field
ElementBased	Element-based format with semantic element extraction

---

ElementType

Semantic element type classification.

Categorizes text content into semantic units for downstream processing. Supports the element types commonly found in Unstructured documents.

Value	Description
Title	Document title
NarrativeText	Main narrative text body
Heading	Section heading
ListItem	List item (bullet, numbered, etc.)
Table	Table element
Image	Image element
PageBreak	Page break marker
CodeBlock	Code block
BlockQuote	Block quote
Footer	Footer text
Header	Header text

---

FormFieldType

Kind of a PDF form field.

Mirrors pdf_oxide’s widget field taxonomy without leaking the upstream type across the binding surface.

Value	Description
Text	Single- or multi-line text input.
Checkbox	Checkbox (on/off toggle).
Radio	Radio-button group member.
Choice	Choice field (dropdown or list box).
Signature	Digital-signature field.
Button	Push button.
Unknown	Field type that could not be classified.

---

FormatMetadata

Format-specific metadata (discriminated union).

Only one format type can exist per extraction result. This provides type-safe, clean metadata without nested optionals.

Value	Description
Pdf	Metadata extracted from a PDF document. — Fields: 0: PdfMetadata
Docx	Metadata extracted from a DOCX Word document. — Fields: 0: DocxMetadata
Excel	Metadata extracted from an Excel spreadsheet. — Fields: 0: ExcelMetadata
Email	Metadata extracted from an email message (EML/MSG). — Fields: 0: EmailMetadata
Pptx	Metadata extracted from a PowerPoint presentation. — Fields: 0: PptxMetadata
Archive	Metadata extracted from an archive (ZIP, TAR, 7Z, etc.). — Fields: 0: ArchiveMetadata
Image	Metadata extracted from a raster or vector image. — Fields: 0: ImageMetadata
Xml	Metadata extracted from an XML document. — Fields: 0: XmlMetadata
Text	Metadata extracted from a plain-text file. — Fields: 0: TextMetadata
Html	Metadata extracted from an HTML document. — Fields: 0: HtmlMetadata
Ocr	Metadata produced by an OCR pipeline. — Fields: 0: OcrMetadata
Csv	Metadata extracted from a CSV or TSV file. — Fields: 0: CsvMetadata
Bibtex	Metadata extracted from a BibTeX bibliography file. — Fields: 0: BibtexMetadata
Citation	Metadata extracted from a citation file (RIS, PubMed, EndNote). — Fields: 0: CitationMetadata
FictionBook	Metadata extracted from a FictionBook (FB2) e-book. — Fields: 0: FictionBookMetadata
Dbf	Metadata extracted from a dBASE (DBF) database file. — Fields: 0: DbfMetadata
Jats	Metadata extracted from a JATS (Journal Article Tag Suite) XML file. — Fields: 0: JatsMetadata
Epub	Metadata extracted from an EPUB e-book. — Fields: 0: EpubMetadata
Pst	Metadata extracted from an Outlook PST archive. — Fields: 0: PstMetadata
Audio	Metadata extracted from an audio or video file. — Fields: 0: AudioMetadata
Code	Code (tree-sitter analyzable source). Carries the structural chunks (function, class, and module boundaries) produced by the tree-sitter extractor, consumed by the chunking pipeline to emit structure-aware Chunks instead of falling back to text-based splitting. Wraps CodeMetadata (a named struct) rather than CodeChunkInfo\[\] directly: FormatMetadata is internally tagged (#\[serde(tag = "format_type")\]), and serde cannot serialize a tagged newtype variant that wraps a sequence — the tag has no map to live in. Wrapping a struct gives serde a map to hold the tag, and keeps this variant shape consistent with every sibling (Variant(XMetadata)) so the derived OpenAPI discriminator can reference a named component schema. — Fields: 0: CodeMetadata

---

CodeDataNodeKind

Discriminates the shape of a CodeDataNode.

Purpose-built mirror of tree_sitter_language_pack.DataNodeKind — kept as an xberg-owned type so binding generators never need to resolve the upstream crate’s types across FFI/language boundaries.

Value	Description
KeyValue	A key/value pair or mapping (JSON/TOML/properties/YAML/HCL/CUE/KDL pair, or a wrapper “object”/“mapping” container).
Element	An XML element with a tag name in key and attributes in attributes.
Sequence	A positional sequence item (JSON array element, YAML block sequence item, CSV/PSV row or cell).

---

TextDirection

Text direction enumeration for HTML documents.

Value	Description
LeftToRight	Left-to-right text direction
RightToLeft	Right-to-left text direction
Auto	Automatic text direction detection

---

LinkType

Link type classification.

Value	Description
Anchor	Anchor link (#section)
Internal	Internal link (same domain)
External	External link (different domain)
Email	Email link (mailto:)
Phone	Phone link (tel:)
Other	Other link type

---

ImageType

Image type classification.

Value	Description
DataUri	Data URI image
InlineSvg	Inline SVG
External	External image URL
Relative	Relative path image

---

StructuredDataType

Structured data type classification.

Value	Description
JsonLd	JSON-LD structured data
Microdata	Microdata
RDFa	RDFa

---

OcrBoundingGeometry

Bounding geometry for an OCR element.

Supports both axis-aligned rectangles (from Tesseract) and 4-point quadrilaterals (from PaddleOCR and rotated text detection).

Value	Description
Rectangle	Axis-aligned bounding box (typical for Tesseract output). — Fields: left: number, top: number, width: number, height: number
Quadrilateral	4-point quadrilateral for rotated/skewed text (PaddleOCR). Points are in clockwise order starting from top-left: \[top_left, top_right, bottom_right, bottom_left\]

---

OcrElementLevel

Hierarchical level of an OCR element.

Maps to Tesseract’s page segmentation hierarchy and provides equivalent semantics for PaddleOCR.

Value	Description
Word	Individual word
Line	Line of text (default for PaddleOCR)
Block	Paragraph or text block
Page	Page-level element

---

PageUnitType

Type of paginated unit in a document.

Distinguishes between different types of “pages” (PDF pages, presentation slides, spreadsheet sheets).

Value	Description
Page	Standard document pages (PDF, DOCX, images)
Slide	Presentation slides (PPTX, ODP)
Sheet	Spreadsheet sheets (XLSX, ODS)

---

RedactionStrategy

Strategy applied when a PII match is rewritten.

Value	Description
Mask	Replace the matched span with a fixed mask token (default "\[REDACTED\]").
Hash	Replace with a SHA-256 hash of the original value (truncated to 16 hex chars). Lets downstream consumers do equality joins without recovering the source.
TokenReplace	Replace with a per-category running token ("\[PERSON_1\]", "\[PERSON_2\]", …) so the same person referenced twice gets the same token within the document.
Drop	Delete the matched span entirely.

---

PiiCategory

PII categories the pattern engine recognises.

Value	Description
Email	Email address (e.g. user@example.com).
Phone	Phone number in any common format.
Ssn	US Social Security Number.
CreditCard	Payment card number (Visa, Mastercard, Amex, etc.).
PostalCode	Postal / ZIP code.
IpAddress	IPv4 or IPv6 address.
Iban	International Bank Account Number.
SwiftBic	SWIFT / BIC bank identifier code.
DateOfBirth	Date of birth.
Person	Person name, surfaced by the optional NER backend.
Organization	Organization name, surfaced by the optional NER backend.
Location	Location, surfaced by the optional NER backend.
Custom	Caller-supplied custom category (e.g. internal employee IDs). Surfaced by the redaction engine when a hit comes from RedactionConfig.custom_terms or RedactionConfig.custom_patterns. The string is the label passed alongside the term/pattern. Use those fields rather than constructing Custom directly via the categories filter — the pattern engine cannot detect arbitrary text from a category name alone. — Fields: 0: string

---

DiffLine

A single line in a unified-diff hunk.

Defined here (rather than only in crate.diff) so RevisionDelta can reference it unconditionally, without requiring the diff Cargo feature. crate.diff re-exports this type verbatim.

Value	Description
Context	Unchanged context line. — Fields: 0: string
Added	Line added in the “after” version. — Fields: 0: string
Removed	Line removed from the “before” version. — Fields: 0: string

---

RevisionKind

Semantic classification of a tracked change.

Value	Description
Insertion	Text or content was inserted.
Deletion	Text or content was deleted.
FormatChange	Run-level formatting (font, size, colour, …) was changed.
Comment	A reviewer comment or annotation.

---

RevisionAnchor

Best-effort document location for a revision.

Value	Description
Paragraph	Body paragraph, identified by its zero-based index in the document flow. — Fields: index: number
TableCell	Cell inside a table. — Fields: row: number, col: number, tableIndex: number
Page	Page, identified by its zero-based index. — Fields: index: number
Slide	Presentation slide, identified by its zero-based index. — Fields: index: number
Sheet	Spreadsheet cell or range, identified by sheet index and optional name. — Fields: index: number, name: string

---

SummaryStrategy

Summarisation strategy.

Value	Description
Extractive	Pure-Rust extractive summary (TextRank over the chunk graph). Deterministic, fast, no external service required.
Abstractive	Abstractive summary produced by liter-llm. Requires liter-llm feature and a configured LlmConfig. Token usage is captured in ExtractedDocument.llm_usage.

---

UriKind

Semantic classification of an extracted URI.

Value	Description
Hyperlink	A clickable hyperlink (web URL, file link).
Image	An image or media resource reference.
Anchor	An internal anchor or cross-reference target.
Citation	A citation or bibliographic reference (DOI, academic ref).
Reference	A general reference (e.g. \ref{} in LaTeX, :ref: in RST).
Email	An email address (mailto: link or bare email).

---

RegionKind

Classification of a detected layout region that warrants VLM extraction.

Each variant maps to a specific prompt optimised for that content type. The mapping is intentionally narrow — only region kinds for which VLM extraction provides a clear quality benefit over classical suppression.

Value	Description
Figure	A figure, diagram, chart, or image region. VLM prompt: describe the diagram / chart, including axis labels, legend entries, and any embedded text.
DenseTable	A densely formatted or complex table that classical extraction garbles. VLM prompt: extract the table as GitHub-Flavoured Markdown.
ComplexLayout	A region whose layout the classical pipeline cannot handle (multi-column insets, heavily annotated forms, mixed text+diagram). VLM prompt: extract all text and structure as markdown, preserving reading order.
Caption	A standalone image to be captioned (not extracted as figure markdown). VLM prompt: produce a single-sentence alt-text-style caption suitable for accessibility tooling and downstream indexing. Used by the captioning post-processor to populate ExtractedImage.caption.

---

EmbeddingsEmbeddingBackend

Inference backend that an EmbeddingPreset runs on.

Onnx presets require the embeddings feature (ONNX Runtime, not available on WASM/Android x86_64 emulator). Static presets require static-embeddings (pure-Rust model2vec inference, no ORT — the only dense-embedding backend available on no-ort-target).

Defaults to Onnx via #[serde(default)] so every existing preset payload (which predates this field) keeps deserializing without change.

Value	Description
Onnx	ONNX Runtime transformer inference (the historical, default backend).
Static	Pure-Rust static (model2vec) inference — no ONNX Runtime.

---

KeywordAlgorithm

Keyword algorithm selection.

Value	Description
Yake	YAKE (Yet Another Keyword Extractor) - statistical approach
Rake	RAKE (Rapid Automatic Keyword Extraction) - co-occurrence based

---

SchemaCompliance

Schema-validation outcome surfaced as one of three buckets.

Fold into the combined confidence score without leaking internal validation error types.

Value	Description
AllValid	Every batch validated against the schema.
PartialValid	At least one batch validated; at least one did not.
AllInvalid	No batch validated.

---

NoChunkingReason

Reason for not chunking a document.

Value	Description
SmallFile	File is below size threshold. — Fields: sizeBytes: number, thresholdBytes: number
FewPages	Document has fewer pages than threshold. — Fields: pageCount: number, threshold: number
TextLayerDetected	PDF has substantial text layer (OCR not needed). — Fields: textCoverage: number, avgCharsPerPage: number
FormatNotChunkable	Document format does not support chunking. — Fields: mimeType: string
ChunkingDisabled	Chunking is disabled by configuration.
FastTextExtraction	Force OCR is disabled and text extraction is fast.

---

ChunkingReason

Reason for chunking a document.

Value	Description
LargeFile	File exceeds size threshold. — Fields: sizeBytes: number, thresholdBytes: number
ManyPages	Document has many pages. — Fields: pageCount: number, threshold: number
OcrRequired	PDF requires OCR and is large. — Fields: pageCount: number, forceOcr: boolean
LargeAndManyPages	Both size and page count exceed thresholds. — Fields: sizeBytes: number, pageCount: number

---

BoundaryReason

Reason for boundary detection.

Value	Description
Start	Start of PDF.
PageOneMarker	Page-one marker (“Page 1”, “1 of N”) detected.
LetterheadReset	Letterhead reset after signature block.
DensityShift	Text density shift with low bigram overlap.
End	End of PDF.

---

PresetCategory

High-level category used to group presets in the registry UI.

Value	Description
Finance	Invoices, receipts, statements, purchase orders, W-9.
Identity	Passports, drivers licenses, insurance cards.
Legal	Contracts, NDAs, agreements.
Logistics	Bills of lading, customs declarations, packing lists.
Medical	Clinical records, lab reports.
Hr	Pay stubs, resumes, employment offers.
Other	Catch-all for documents that don’t fit the other categories.

---

PsmMode

Page Segmentation Mode for Tesseract OCR.

Value	Description
OsdOnly	Orientation and script detection only.
AutoOsd	Automatic page segmentation with OSD.
AutoOnly	Automatic page segmentation without OSD or OCR.
Auto	Fully automatic page segmentation with no OSD (default).
SingleColumn	Assume a single column of text of variable sizes.
SingleBlockVertical	Assume a single uniform block of vertically aligned text.
SingleBlock	Assume a single uniform block of text.
SingleLine	Treat the image as a single text line.
SingleWord	Treat the image as a single word.
CircleWord	Treat the image as a single word in a circle.
SingleChar	Treat the image as a single character.

---

ProbeStatus

Outcome of a single doctor check.

Value	Description
Pass	The backend or setting will work as configured.
Warn	The check ran and found something actionable, but nothing is broken (e.g. stray cache files, stale model revisions). Never fails the report.
Fail	The configured setup will not work (or will silently degrade) on this host.
Skip	The check cannot run locally (e.g. model not cached, feature not compiled in); first real use decides, possibly after a download.

---

PaddleInferenceBackend

Which concrete ONNX inference engine PaddleOCR model loading uses.

Mirrors sceptre.Backend for the PaddleOCR backend: Ort is the native, full-featured path (acceleration/execution-provider hook, ONNX-embedded dictionary metadata); Tract is the pure-Rust, CPU-only path used on targets where ort cannot link (Android x86_64 emulator, WASM once wired).

Value	Description
Ort	Native ONNX Runtime (requires the paddle-ocr-ort feature).
Tract	Pure-Rust ONNX via tract (requires the paddle-ocr-tract feature).

---

PaddleLanguage

Supported languages in PaddleOCR.

Maps user-friendly language codes to paddle-ocr-rs language identifiers.

Value	Description
English	English
Chinese	Simplified Chinese
Japanese	Japanese
Korean	Korean
German	German
French	French
Latin	Latin script (covers most European languages)
Cyrillic	Cyrillic (Russian and related)
TraditionalChinese	Traditional Chinese
Thai	Thai
Greek	Greek
EastSlavic	East Slavic (Russian, Ukrainian, Belarusian)
Arabic	Arabic (Arabic, Persian, Urdu)
Devanagari	Devanagari (Hindi, Marathi, Sanskrit, Nepali)
Tamil	Tamil
Telugu	Telugu

---

LayoutClass

The 18 canonical document layout classes.

All model backends (RT-DETR, YOLO, etc.) map their native class IDs to this shared set. Models with fewer classes (DocLayNet: 11, PubLayNet: 5) map to the closest equivalent.

Wire format is snake_case in all serializers (JSON, TOML, YAML).

Value	Description
Caption	Figure or table caption text.
Chart	Chart or graph visualization.
Footnote	Footnote or endnote text.
Formula	Mathematical formula or equation.
ListItem	A single item in a bulleted or numbered list.
PageFooter	Running footer at the bottom of a page.
PageHeader	Running header at the top of a page.
Picture	Image, chart, or other graphical element.
SectionHeader	Section heading.
Table	Data table.
Text	Body text paragraph.
Title	Document or chapter title.
DocumentIndex	Table of contents or index.
Code	Source code block.
CheckboxSelected	Checkbox in selected state.
CheckboxUnselected	Checkbox in unselected state.
Form	Form field or form element.
KeyValueRegion	Key-value pair region (e.g. label + value in a form).

---

BrowserMode

When to use the headless browser fallback.

Value	Description
Auto	Automatically detect when JS rendering is needed and fall back to browser.
Always	Always use the browser for every request.
Never	Never use the browser fallback.
Stealth	Always use the browser with all stealth surfaces enabled. Behaves like Always for escalation purposes (every request is routed through the browser tier), but additionally enables: - browser JavaScript stealth patches - native-backend TLS fingerprint spoofing - stealth-aware default user-agent when no explicit UA is set - 1920×1080 viewport override Use this instead of setting the now-removed BrowserConfig.stealth boolean field.

---

BrowserWait

Wait strategy for browser page rendering.

Value	Description
NetworkIdle	Wait until network activity is idle.
Selector	Wait for a specific CSS selector to appear in the DOM.
Fixed	Wait for a fixed duration after navigation.

---

BrowserBackend

Browser backend used for JavaScript rendering.

Value	Description
Chromiumoxide	Existing Chromium/CDP backend powered by chromiumoxide.
Native	Crawlberg-owned native browser backend derived from Obscura.

---

AuthConfig

Authentication configuration.

Value	Description
Basic	HTTP Basic authentication. — Fields: username: string, password: string
Bearer	Bearer token authentication. — Fields: token: string
Header	Custom authentication header. — Fields: name: string, value: string

---

AssetCategory

The category of a downloaded asset.

Value	Description
Document	A document file (PDF, DOC, etc.).
Image	An image file.
Audio	An audio file.
Video	A video file.
Font	A font file.
Stylesheet	A CSS stylesheet.
Script	A JavaScript file.
Archive	An archive file (ZIP, TAR, etc.).
Data	A data file (JSON, XML, CSV, etc.).
Other	An unrecognized asset type.

---

PreprocessingPreset

HTML preprocessing aggressiveness level.

Controls the extent of cleanup performed before conversion. Higher levels remove more elements.

Value	Description
Minimal	Minimal cleanup. Remove only essential noise (scripts, styles).
Standard	Standard cleanup. Default. Removes navigation, forms, and other auxiliary content.
Aggressive	Aggressive cleanup. Remove extensive non-content elements and structure.

---

HeadingStyle

Heading style options for Markdown output.

Controls how headings (h1-h6) are rendered in the output Markdown.

Value	Description
Underlined	Underlined style (=== for h1, — for h2).
Atx	ATX style (# for h1, ## for h2, etc.). Default.
AtxClosed	ATX closed style (# title #, with closing hashes).

---

ListIndentType

List indentation character type.

Controls whether list items are indented with spaces or tabs.

Value	Description
Spaces	Use spaces for indentation. Default. Width controlled by list_indent_width.
Tabs	Use tabs for indentation.

---

WhitespaceMode

Whitespace handling strategy during conversion.

Determines how sequences of whitespace characters (spaces, tabs, newlines) are processed.

Value	Description
Normalized	Collapse multiple whitespace characters to single spaces. Default. Matches browser behavior.
Strict	Preserve all whitespace exactly as it appears in the HTML.

---

NewlineStyle

Line break syntax in Markdown output.

Controls how soft line breaks (from <br> or line breaks in source) are rendered.

Value	Description
Spaces	Two trailing spaces at end of line. Default. Standard Markdown syntax.
Backslash	Backslash at end of line. Alternative Markdown syntax.

---

CodeBlockStyle

Code block fence style in Markdown output.

Determines how code blocks (<pre><code>) are rendered in Markdown.

Value	Description
Indented	Indented code blocks (4 spaces). CommonMark standard.
Backticks	Fenced code blocks with triple backticks. Default (GFM). Supports language hints.
Tildes	Fenced code blocks with tildes (~~~). Supports language hints.

---

HighlightStyle

Highlight rendering style for <mark> elements.

Controls how highlighted text is rendered in Markdown output.

Value	Description
DoubleEqual	Double equals syntax (==text==). Default. Pandoc-compatible.
Html	Preserve as HTML (==text==). Original HTML tag.
Bold	Render as bold (text). Uses strong emphasis.
None	Strip formatting, render as plain text. No markup.

---

LinkStyle

Link rendering style in Markdown output.

Controls whether links and images use inline [text](url) syntax or reference-style [text][1] syntax with definitions collected at the end.

Value	Description
Inline	Inline links: \[text\](url). Default.
Reference	Reference-style links: \[text\]\[1\] with \[1\]: url at end of document.

---

UrlEscapeStyle

URL encoding strategy for link and image destinations.

Controls how special characters in URL destinations are handled when they require escaping to produce valid Markdown.

The Angle variant (default) wraps the destination in angle brackets: [text](<url with spaces>). This is the CommonMark-specified escape hatch but breaks when the URL itself contains >.

The Percent variant percent-encodes every character that is not an RFC 3986 unreserved character or /, producing a destination safe for all Markdown parsers: [text](url%20with%20spaces).

Value	Description
Angle	Wrap destinations that contain spaces or newlines in angle brackets. Default.
Percent	Percent-encode all characters that are not RFC 3986 unreserved or /.

---

Errors

XbergError

Main error type for all Xberg operations.

All errors in Xberg use this enum, which preserves error chains and provides context for debugging.

Variants

• Io - File system and I/O errors (always bubble up)
• Parsing - Document parsing errors (corrupt files, unsupported features)
• Ocr - OCR processing errors
• Validation - Input validation errors (invalid paths, config, parameters)
• Cache - Cache operation errors (non-fatal, can be ignored)
• ImageProcessing - Image manipulation errors
• Serialization - JSON/MessagePack serialization errors
• MissingDependency - Missing optional dependencies (tesseract, etc.)
• Plugin - Plugin-specific errors
• LockPoisoned - Mutex/RwLock poisoning (should not happen in normal operation)
• UnsupportedFormat - Unsupported MIME type or file format
• Other - Catch-all for uncommon errors

Errors are thrown as plain Error objects with descriptive messages.

Variant	Description
Io	A file system or I/O operation failed. These errors always bubble up unchanged.
Parsing	Document parsing failed (e.g. corrupt file, unsupported format feature).
Ocr	An OCR engine returned an error or produced unusable output.
Validation	Invalid configuration or input parameters were supplied.
Cache	A cache read or write operation failed.
ImageProcessing	An image manipulation operation (resize, decode, DPI conversion) failed.
Serialization	JSON or MessagePack serialization/deserialization failed.
MissingDependency	A required optional system dependency (e.g. tesseract) was not found.
Plugin	A registered plugin returned an error during extraction.
LockPoisoned	An internal Mutex or RwLock was found in a poisoned state.
UnsupportedFormat	The document’s MIME type is not supported by any registered extractor.
Embedding	The embedding model or embedding pipeline returned an error.
Reranking	The reranker model or reranking pipeline returned an error. Since v5.0.
Transcription	Audio/video transcription failed.
Timeout	The extraction operation exceeded the configured time limit.
Cancelled	The extraction was cancelled via a CancellationToken.
Security	A security policy was violated (e.g. zip bomb, oversized archive).
Other	A catch-all for uncommon errors that do not fit another variant.

---

HeuristicsError

Errors that can occur during heuristics analysis.

Errors are thrown as plain Error objects with descriptive messages.

Variant	Description
ConfigError	Invalid configuration value.
PdfAnalysisError	PDF analysis step failed (only when heuristics-pdf feature is active).

---

LoadError

Errors produced while loading or validating a preset file.

Errors are thrown as plain Error objects with descriptive messages.

Variant	Description
Parse	The file is not valid JSON.
SchemaValidation	The file parses as JSON but does not validate against the meta-schema.
Deserialize	The file validates but cannot be deserialized into Preset.
IdMismatch	The preset’s declared id does not match its file-system location.
BadMetaSchema	The meta-schema itself failed to compile.
Io	A filesystem I/O error occurred while reading a preset directory.

---

ResolveError

Errors produced while resolving a preset against caller overrides.

Errors are thrown as plain Error objects with descriptive messages.

Variant	Description
SchemaNotObject	A custom schema override was supplied but is not a JSON object.

---