If you parsed a document this week — with Docling or anything else — the next decision is how to cut it into chunks before you embed it. That decision quietly sets two things: how well retrieval works, and how heavy your ingestion container gets. Here's the short answer, then the code.

The short answer: if chunk quality or image size matters, install Chonkie — it's a 505KB wheel, it's fast, and semantic and late chunking are built in. If neither matters much, use the splitter that already ships with LangChain or LlamaIndex and save yourself a dependency. All three are real, maintained, and free.

What chunking actually decides#

Retrieval only ever returns chunks, never documents. If a chunk splits a table in half, mixes two topics, or drops the sentence that gave a number its meaning, no reranker downstream fully recovers it. So the chunker is not plumbing — it's the last place you shape what the model is allowed to see. The three libraries below take three different bets on how much effort that's worth.

Chonkie: the dedicated chunker#

Chonkie (MIT, maintained by chonkie-inc) does one thing: chunk text for RAG. That focus buys you a 505KB wheel versus 1–12MB for the framework alternatives — real savings in a cold-start-sensitive ingestion image — and, in its own benchmarks, ~33× faster token chunking and roughly 2× faster sentence chunking than the slowest competitors. It ships ten chunker types: Token, Sentence, Recursive, Semantic, SDPM (semantic double-pass merge), Late, Code, Neural, Slumber, and Table.

The API is a two-liner, and it's token-aware by default:

from chonkie import TokenChunker

chunker = TokenChunker(chunk_size=512, chunk_overlap=64)
chunks = chunker.chunk(text)

for c in chunks:
    print(c.token_count, c.start_index, c.end_index)
    embed(c.text)

For quality-sensitive corpora, swap in the SemanticChunker, which cuts at topical shifts using embedding similarity instead of a fixed token count:

from chonkie import SemanticChunker

chunker = SemanticChunker(
    embedding_model="minishlab/potion-base-8M",  # small, fast, local
    threshold=0.5,
    chunk_size=512,
)
chunks = chunker.chunk(text)

Under the hood Chonkie organizes ingestion as a CHONK pipeline — Chefs preprocess, Chunkers split, Refineries enhance (e.g., add overlap or embeddings), and Porters export — so you can grow from "just chunk this string" to a full pipeline without changing libraries. It supports 16+ embedding providers and 5+ tokenization methods, so it slots onto whatever embedding model you already pay for.

LangChain: the one you already have#

If you run LangChain, you already have RecursiveCharacterTextSplitter, and it's a perfectly good default. It splits hierarchically on a list of separators (paragraphs, then lines, then words) until chunks fit:

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512, chunk_overlap=64,
)
chunks = splitter.split_text(text)  # list[str]

The catch worth knowing: it counts characters by default, not tokens. Pass a length_function backed by your tokenizer if chunk size needs to track your embedding model's context, or you'll size chunks by a proxy that drifts on code and non-English text. For most teams already inside LangChain, that one adjustment is cheaper than adding a dependency.

LlamaIndex: the one that speaks Nodes#

LlamaIndex's SentenceSplitter keeps sentences intact and is the natural pick if your pipeline already passes Node objects around:

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=512, chunk_overlap=64)
nodes = splitter.get_nodes_from_documents(documents)  # list[Node]

For semantic cuts, LlamaIndex has SemanticSplitterNodeParser, which — like Chonkie's SemanticChunker — spends an embedding pass to break at topical shifts. If you're already in LlamaIndex, staying there keeps your metadata and node IDs consistent through the rest of the pipeline.

How to actually choose#

The mistake isn't picking the "wrong" library — all three are fine. The mistake is never measuring, shipping fixed 1000-character chunks, and blaming the model for retrieval it was never given a fair shot at. Chunk, measure recall, tune size and overlap, and only then decide whether semantic chunking earns its embedding pass.

Next in the pipeline: once chunks are sized, how you order them in the prompt and which chunking strategy fits your corpus are the two follow-on knobs that move retrieval the most.