Embeddings Overview

Overview of the different embeddings available in Chonkie

Chonkie provides a variety of embeddings handlers to handle different embedding models in a consistent manner. Embeddings handlers are used in conjunction with chunkers to embed chunks of text. Only few chunkers require embeddings, see the Chunkers Overview for more information.

Installation

Embeddings handlers require additional dependencies. See the Installation Guide for more information.

By default, Chonkie semantic installation includes Model2VecEmbeddings, which is the current default embeddings handler

Available Embeddings

AutoEmbeddings

Automatically select the best embeddings handler for your use case.

SentenceTransformerEmbeddings

Embed text using SentenceTransformer embeddings (requires sentence-transformers).

OpenAIEmbeddings

Embed text using OpenAI embeddings (requires openai).

Model2VecEmbeddings

Embed text using Model2Vec embeddings (requires model2vec).

Common Interface

All embeddings handlers share a consistent interface:

# Single text embedding
emb = embeddings.embed(text)

# Batch processing
emb = embeddings.embed_batch(texts)

# Direct calling
emb = embeddings(text)  # or embeddings([text1, text2])