Chunking

What is chunking?

Handing a fifty-page document to an AI in one giant blob is expensive and easy to get wrong. Chunking means splitting that long document into small pieces, each carrying enough context to be found and understood on its own.

Why chunk — and why do it carefully?

Too long, and the model drowns
The context window is finite. Dump a whole book in and the important parts get diluted.
Too short, and meaning breaks
A crude split can chop a sentence in half or separate a pronoun from what it points to.

Common ways to slice

Fixed-size chunks
Cut by character or token count. Simple, but it can slice straight through an idea.
Structural chunks
Follow headings, paragraphs and sections, matching how the document is actually built.
Overlapping chunks
Leave a little overlap between neighbors so nothing slips through the seams.

Why it matters so much for RAG

Retrieval-augmented generation lives or dies by how well you chunk. Cut well and you get accurate recall and grounded answers; cut badly and even a strong model can't make sense of it.

Bottom line: chunking cuts long documents into bite-sized pieces — feed them well and the AI digests them well.