Get Started with Chonkie

Get started with Chonkie

Using Chonkie takes two simple steps: First, install the package. Next, start chunking!

This page covers Chonkie Open Source. To get started with our API, visit the API Reference.

Installation

Python

pip

pip install chonkie

Want more features? Install everything with pip install "chonkie[all]". See Installation for more options.

JavaScript

Install the core package for local chunking

npm

npm install @chonkiejs/core

Chonkie JS provides local support for TokenChunker, SentenceChunker, RecursiveChunker, FastChunker, TableChunker, SemanticChunker, and CodeChunker. Other chunkers are available through the API.

CHONK! 🦛✨

Python

# First import the chunker you want from Chonkie
from chonkie import TokenChunker

# Initialize the chunker
chunker = TokenChunker() # defaults to using character tokenizer

# Here's some text to chunk
text = """Woah! Chonkie, the chunking library is so cool!"""

# Chunk some text
chunks = chunker(text)

# Access chunks
for chunk in chunks:
  print(f"Chunk: {chunk.text}")
  print(f"Tokens: {chunk.token_count}")