The no-nonsense RAG chunking library that’s lightweight, fast, and ready to CHONK your texts!
Ever found yourself making a RAG pipeline yet again (your 2,342,148th one), only to realize you’re stuck having to write chunking with bloated software library X or the painfully feature-less library Y?
WHY CAN’T THIS JUST BE SIMPLE, UGH?Well, look no further than Chonkie TS! (chonkie boi is a gud boi 🦛)
Feature-rich
All the CHONKs you’d ever need for your RAG applications
Easy to use
Install, Import, CHONK - it’s that simple!
Lightning Fast
CHONK at the max speed of TypeScript! tssssooooooom
Type-Safe
Full TypeScript support with type safety
Lightweight
No bloat, just CHONK - minimal dependencies by default
Get started with Chonkie TS in three simple steps: Install, Import and CHONK!
Copy
Ask AI
npm install chonkie
Want minimal dependencies? Install with optional dependencies omitted:
Copy
Ask AI
npm install chonkie --omit=optional
Chonkie TS follows a special approach to dependencies, keeping the base installation lightweight while allowing you to add extra features as and when needed.
Please check the Installation page for more details.
Release the CHONK! 🦛✨
Copy
Ask AI
import { TokenChunker } from 'chonkie';async function main() { // Create a token chunker with default options const chunker = await TokenChunker.create(); // Here's some text to chunk const text = "Woah! Chonkie is such a great ts library!"; // Chunk some text const chunks = await chunker.chunk(text); // Access chunks for (const chunk of chunks) { console.log(chunk.text); console.log(chunk.tokenCount); }}main();
Chonkie TS is a port of the original Python library, designed specifically for TypeScript developers. While it maintains the core functionality, it adds type safety and TypeScript-specific features. The library is under active development and working towards feature parity with the original Python version.