What you'll learn: what chunk size and overlap do in the capstone's heading-aware chunker, how to measure them on a fixed question set, and why on this corpus size barely matters while removing headings does.
At a glance
Figure 1. A chunking decision, in the order to ask the questions
How to read this: the first two questions are about the documents, not about numbers, and they settle most cases before any sweep is run.
- A chunk is the unit that gets scored, shown to the model and cited.
- Chunk size is a no-op on this corpus from 120 words up: no section is longer than 99 words, and the chunker never merges across headings.
- Overlap costs storage; on this corpus it did not change recall.
- Measure on a fixed question set; do not tune by one good answer.
Scenario
Composite scenario, not a real company. Lumen Industrial Supply, a fictional distributor, has an assistant that answers from 30 short product documents. One colleague says "bigger chunks give the model more context"; another says "smaller chunks are more precise". Both argue from habit. The team has a 36-question golden set and a retriever it can rebuild in a second, so a table can settle it. The numbers come from running it, except the cost example.
What you will be able to do
- Explain how
chunk_documentforms windows from whole lines and where overlap enters. - Run a size and overlap sweep and read recall, chunk count and context words together.
- Predict when changing chunk size is a no-op.
- Choose a chunking approach for a new document type and name what to measure.
Prerequisites
- Document ingestion and metadata for RAG: the
DocumentandChunktypes. - Theory: Data and retrieval. This lesson only measures.
Core ideas
How the capstone forms a chunk
The chunker splits a document at headings (a chunk never crosses one). Inside a section it adds whole lines to a window until the next would exceed the size, then starts a new window that repeats trailing lines worth at most overlap words.
Verified: capstone/fde-product-discovery-copilot/app/retrieval/chunking.py
def _windows(lines: list[str], size: int, overlap: int) -> list[list[str]]:
"""Group whole lines into windows of about `size` words; the next window repeats trailing
lines worth at most `overlap` words."""
windows: list[list[str]] = []
current: list[str] = []
count = 0
for line in lines:
n = len(line.split())
if current and count + n > size:
windows.append(current)
current, count = [], 0
for prev in reversed(windows[-1]): # carry the overlap
pn = len(prev.split())
if count + pn > overlap:
break
current.insert(0, prev)
count += pn
current.append(line)
count += n
if current:
windows.append(current)
return windowsBecause it groups whole lines, a single 300-word paragraph is never split: at size 120 it stays one 300-word chunk. A test in the lesson folder pins this.
Figure 2. How size and overlap change the windows
| Setting | Windows over six 5-word lines (L1 to L6) | Chunks | Words stored |
|---|---|---|---|
| size 10, overlap 0 | L1 L2 / L3 L4 / L5 L6 | 3 | 30 |
| size 10, overlap 5 | L1 L2 / L2 L3 / L3 L4 / L4 L5 / L5 L6 | 5 | 50 |
How to read this: with overlap 5 each window repeats the last line of the one before, so you store 50 words for 30 words of text.
Overlap buys one thing: an answer that straddles a boundary appears whole in one chunk. It costs storage and duplicate hits.
Predict first
Predict first. The corpus has 174 sections. The capstone default is size 120 and overlap 20. You raise the size to 480 and rerun the 23 golden questions that expect documents. Does document recall at 3 go up, down, or stay the same?
Answer
It stays the same, to the last decimal. The longest section is 99 words, so at size 120 every section is already one chunk, and sections are never merged across headings. The sweep below shows 174 chunks at both settings.
Estimating cost with a model, then checking it
The widget below models fixed windows over a document with no headings. Take the longest document, compat-opcua, at about 236 words, with windows of 60 words overlapping by 10, and count the top 3 chunks.