Retrieval

Pick a retrieval mode, generate the vectors it needs, and combine scores yourself — or let Reciprocal Rank Fusion do it for you. See Retrieval Modes for when to use each primitive. Standalone functions shown below (embed_texts, embed_sparse, embed_multi_vector, rerank) are Rust-native entry points; consult each language’s API reference for the exact bound name in that language.

Dense embeddings

Call embed_texts (or embed_texts_async) with an EmbeddingConfig to get one vector per input string.

use xberg::{embed_texts, EmbeddingConfig, EmbeddingModelType};

let config = EmbeddingConfig {
    model: EmbeddingModelType::Preset { name: "gte-modernbert-base".to_string() },
    ..Default::default()
};

let vectors = embed_texts(
    vec!["first document".to_string(), "second document".to_string()],
    &config,
)?;
# Ok::<(), xberg::XbergError>(())

For asymmetric presets (arctic-embed-m-v2.0), call embedding_query_prefix(&config) and prepend the returned prefix to query text only — document text is embedded verbatim. Symmetric presets return None.

Compare two dense vectors by cosine similarity or dot product for ranking. See Embeddings for chunking integration, custom/LLM/plugin model variants, and language-specific examples.

Sparse embeddings (SPLADE)

Call embed_sparse (or embed_sparse_async) with a SparseEmbeddingConfig. Each result is a SparseEmbedding { indices, values } pair — the non-zero vocabulary weights for that text.

use xberg::{embed_sparse, SparseEmbeddingConfig, SparseEmbeddingModelType};

let config = SparseEmbeddingConfig {
    model: SparseEmbeddingModelType::Preset { name: "opensearch-v3-distill".to_string() },
    ..Default::default()
};

let sparse_vectors = embed_sparse(
    vec!["first document".to_string(), "second document".to_string()],
    &config,
)?;
# Ok::<(), xberg::XbergError>(())

Score two sparse vectors with a dot product over their shared (overlapping) indices — index a SparseEmbedding the same way you would BM25 postings.

ColBERT late-interaction

Call embed_multi_vector (or embed_multi_vector_async) with a LateInteractionConfig. Set is_query to true for query text (applies [Q] marker insertion and query-augmentation padding) and false for document text ([D] marker, no padding). Each result is a MultiVectorEmbedding { num_tokens, dim, data } — a row-major buffer of per-token vectors.

use xberg::{embed_multi_vector, LateInteractionConfig, LateInteractionModelType, max_sim_score};

let config = LateInteractionConfig {
    model: LateInteractionModelType::Preset { name: "gte-moderncolbert".to_string() },
    ..Default::default()
};

let query_mv = &embed_multi_vector(
    vec!["how to train a dog".to_string()],
    &config,
    true,
)?[0];
let doc_mv = &embed_multi_vector(
    vec!["dog training requires patience".to_string()],
    &config,
    false,
)?[0];

let score = max_sim_score(query_mv, doc_mv);
# Ok::<(), xberg::XbergError>(())

max_sim_score computes MaxSim directly: for each query token row, the maximum dot product over all document token rows, summed across query rows. Use max_sim_rank to rank a candidate set of document multi-vectors against one query multi-vector in a single call.

Reranking

Once you have a candidate set from any retrieval mode above, sharpen it with a cross-encoder:

use xberg::{rerank, RerankerConfig, RerankerModelType};

let config = RerankerConfig {
    model: RerankerModelType::Preset { name: "ettin-reranker-150m".to_string() },
    ..Default::default()
};

let results = rerank(
    "how to train a dog".to_string(),
    candidate_documents,
    &config,
)?;
# Ok::<(), xberg::XbergError>(())

See Reranking for the full preset catalog, custom HuggingFace models, and Python/TypeScript/Go/Java examples.

Hybrid retrieval (manual RRF)

Xberg’s embedding, sparse, and late-interaction primitives are unopinionated about storage — bring your own vector index, inverted index, or database. To combine dense, full-text, and sparse rankings into one result set, apply Reciprocal Rank Fusion yourself over each arm’s ranked candidate list:

def rrf_fuse(ranked_lists: list[list[str]], k: int = 60) -> dict[str, float]:
    """ranked_lists: one ranked list of document ids per retrieval arm."""
    scores: dict[str, float] = {}
    for ranked_ids in ranked_lists:
        for rank, doc_id in enumerate(ranked_ids):
            scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank + 1)
    return scores

dense_ranked = ["doc3", "doc1", "doc7"]      # from cosine similarity over dense vectors
sparse_ranked = ["doc1", "doc9", "doc3"]     # from sparse dot product
fulltext_ranked = ["doc1", "doc3", "doc2"]   # from your full-text index

fused = rrf_fuse([dense_ranked, sparse_ranked, fulltext_ranked])
top_k = sorted(fused, key=fused.get, reverse=True)[:10]

Each arm contributes 1 / (k + rank + 1) per document, where rank is the document’s 0-indexed position within that arm’s own ranked list; documents absent from an arm simply don’t contribute for it. This needs no per-arm score normalization, which is what makes RRF a robust way to combine otherwise incomparable scales (cosine similarity, sparse dot products, BM25-style text scores).

Next steps

• Retrieval Modes — when to use each primitive
• Embeddings — dense embedding configuration, chunking, plugin backends
• Reranking — cross-encoder presets, custom models, async execution
• Model Sources — upstream sources and licenses for every model