Fixed-Size Document Chunking with Overlap

The Core Idea: Slicing Text with Overlap

Fixed-size chunking is exactly what it sounds like: we slice a document into segments of a predetermined length, usually measured in characters or tokens. It's the simplest method because it doesn't try to understand the document's structure; it just cuts.

However, this simplicity comes with a risk. A rigid cut can happen mid-sentence or even mid-word, destroying the semantic context you're trying to preserve for the embedding model. To mitigate this, we introduce chunk overlap. This means that a small portion of text from the end of one chunk is repeated at the beginning of the next. This simple technique significantly increases the chance that a complete thought or sentence is captured fully within at least one chunk.

The image below illustrates this concept perfectly. A piece of text is split into three chunks, with a small overlap ensuring the context around the split points is preserved.

A visual representation of fixed-size chunking. The text is divided into three chunks of a specified size (35 characters), with a 4-character overlap between adjacent chunks to maintain context.

The following article provides a concise definition and a useful table comparing fixed-size chunking to other methods.

Implementing Fixed-Size Chunking

As a software engineer, you'll find the algorithm for fixed-size chunking quite straightforward. It operates on two main parameters: chunk_size and overlap. The key is to determine the "step size"—how far to advance the starting point for each new chunk. This is simply chunk_size - overlap.

The process is as follows:

1. Start at the beginning of the text (index 0).
2. Take a slice of length chunk_size. This is your first chunk.
3. Advance the starting point by step_size.
4. Take another slice of length chunk_size. This is your second chunk.
5. Repeat until you have processed the entire text.

The article from OneUptime not only explains this algorithm but also provides a complete, from-scratch Python implementation that is perfect for understanding the mechanics.

Now, let's see this FixedSizeChunker class in action. The code below demonstrates how to initialize the chunker and use it to split a sample document.

# Assuming the FixedSizeChunker class from the article is defined in a file 
# named fixed_size_chunker.py

from fixed_size_chunker import FixedSizeChunker

sample_text = """
Retrieval-Augmented Generation (RAG) is a technique that enhances
large language models by providing them with relevant context from
external knowledge bases. The process involves three main steps:
first, documents are split into chunks and embedded into vectors;
second, user queries are embedded and used to find similar chunks;
third, retrieved chunks are included in the LLM prompt to provide
context for generating accurate responses.
"""

# Initialize the chunker with a chunk size of 150 characters and an overlap of 30.
chunker = FixedSizeChunker(chunk_size=150, overlap=30)

# Generate chunks from the text.
char_chunks = chunker.chunk_by_characters(sample_text)

# Print the results
for chunk in char_chunks:
    print(f"\n--- Chunk {chunk.chunk_index} ---")
    print(f"Position: {chunk.start_index}-{chunk.end_index}")
    print(f"Length: {len(chunk.text)} chars")
    print(f"Text: '{chunk.text}'")

# --- Expected Output ---
#
# --- Chunk 0 ---
# Position: 1-151
# Length: 150 chars
# Text: 'Retrieval-Augmented Generation (RAG) is a technique that enhances
# large language models by providing them with relevant context from
# external knowledg'
#
# --- Chunk 1 ---
# Position: 121-271
# Length: 150 chars
# Text: 'e models by providing them with relevant context from
# external knowledge bases. The process involves three main steps:
# first, documents are split into chu'
#
# --- Chunk 2 ---
# Position: 241-391
# Length: 150 chars
# Text: 's are split into chunks and embedded into vectors;
# second, user queries are embedded and used to find similar chunks;
# third, retrieved chunks are include'
# ... and so on

Notice how the overlapping text (e.g., "e models by providing...") appears at the end of Chunk 0 and the beginning of Chunk 1, successfully bridging the context.

While building from scratch is excellent for understanding, in practice, you'll often use libraries like LangChain or LlamaIndex. The following video clip shows how to use LangChain's CharacterTextSplitter to achieve the same result.

Choosing Your Chunk Size and Overlap

You now know how to implement fixed-size chunking, but how do you choose the right chunk_size and overlap? These are not arbitrary numbers; they are crucial hyperparameters that directly affect your RAG system's performance.

The Overlap Strategy

A common rule of thumb is to set the overlap to be 10-20% of the chunk size. This usually provides a good balance between context preservation and data redundancy. The OneUptime article has an excellent section with visuals that compare different overlap percentages.

The Chunk Size Dilemma: Characters vs. Tokens

Choosing the chunk_size is even more critical. Your first instinct might be to pick a number of characters, like 200 or 500. However, there's a major pitfall here. The embedding models we use don't see characters; they see tokens. Each embedding model has a strict maximum sequence length (a token limit).

If you create a character-based chunk that, when tokenized, exceeds this limit, the model will simply truncate the input. All the text beyond the limit will be ignored during embedding, meaning that information becomes invisible to your retrieval system. This is a subtle but common failure mode in RAG pipelines.

The following video explains this problem exceptionally well and shows why defining chunk size in terms of tokens is a more robust approach.

This insight—that chunking should ideally be coupled to the tokenizer of your chosen embedding model—is a key step toward building a more reliable and performant RAG system.

Conclusion

In this lesson, you've moved from theory to practice by implementing the first and most fundamental chunking strategy. You now have a tangible method for breaking down large documents into pieces suitable for a RAG pipeline.

Here are the key takeaways:

• Fixed-size chunking is a simple, fast, and deterministic method for splitting text based on a fixed length (e.g., characters).
• Chunk overlap is essential to mitigate the risk of splitting sentences or ideas, preserving context across chunk boundaries.
• The core algorithm iterates through text with a step size calculated as chunk_size - overlap.
• A major pitfall is that character-based chunk_size does not directly correspond to tokens. Exceeding an embedding model's token limit will lead to truncation and information loss.
• For more robust systems, it is better to define chunk sizes in terms of tokens rather than characters.

Fixed-size chunking is a valuable tool and an excellent starting point. However, its disregard for document structure can be a limitation. In our next lesson, we will explore a more sophisticated technique: recursive character-based chunking. This method attempts to split text along natural semantic boundaries like paragraphs and sentences, leading to more coherent chunks.