Parent-Child Chunks

The core feature enabling retrieval at different levels of detail. A query can retrieve a high-level parent chunk (e.g., a full section) for broad context or a specific child chunk (e.g., a paragraph) for precise information. This allows the system to adapt to query ambiguity—returning a parent for a general question and a child for a specific fact. The retrieval engine can score and return chunks from either level based on semantic similarity.

Each child chunk is explicitly linked to its parent. When a child chunk is retrieved for its precise relevance, the system can automatically include the content of its parent chunk to provide necessary surrounding context. This mitigates the context fragmentation problem of flat chunking, where a retrieved sentence may lack the introductory definitions or preceding arguments needed for the LLM to interpret it correctly. The link acts as a deterministic path to expand context on-demand.

Different embedding models can be used for parents and children to optimize for their distinct characteristics. For example:

• Children are embedded with models fine-tuned for sentence or short-paragraph similarity (e.g., all-MiniLM-L6-v2).
• Parents can be embedded with models better suited for longer passages or with a separate model to summarize the parent's content into a dense vector. This allows the retrieval system to perform a hybrid search, querying both embedding spaces and merging results.

Compared to simple chunk overlap, which creates many redundant, slightly offset chunks, parent-child structuring is more storage-efficient. Overlap creates N chunks with repeated text. A parent-child hierarchy creates P parents + C children, where C is typically less than the total overlapping chunks needed for equivalent coverage. This reduces index bloat in the vector database, lowering storage costs and potentially improving query latency by searching a smaller, more structured corpus.

Child chunks automatically inherit metadata from their parent (e.g., document title, author, section number). This enables powerful metadata filtering during retrieval. A query can be scoped to "find child chunks about quantum entanglement only within parent chunks where document_type = 'research_paper'." This provides a structured way to combine semantic search with faceted filtering, greatly improving precision in enterprise corpora with rich metadata.

Major RAG frameworks provide native support for this pattern:

• LlamaIndex: Uses HierarchicalNodeParser to create ParentDocumentNode and ChildDocumentNode objects, with built-in retrieval strategies like AutoMergingRetriever.
• LangChain: Achieves this via the ParentDocumentRetriever, which stores small chunks (children) with embeddings but associates them with larger source documents (parents) for retrieval. These implementations handle the mechanics of splitting, linking, and the retrieval logic, allowing engineers to focus on tuning granularity.

In legal RAG systems, a contract is a parent chunk. Its children are granular clauses: indemnification, termination, liability caps. A query like "What are the termination conditions?" retrieves the specific child chunk for high precision. A broader query like "Summarize this agreement" retrieves the parent for comprehensive context, ensuring all key clauses are considered together.

For developer assistance, a class or module overview serves as the parent chunk. Its children are individual method signatures, parameter descriptions, and code examples. A precise query ("What arguments does model.predict() accept?") fetches the exact child. A novice's query ("How do I use this library?") retrieves the parent overview first, providing the necessary foundational context before drilling down.

A research paper's abstract is a parent chunk summarizing the entire work. Children represent individual sections: Introduction, Methodology, Results, Discussion. This allows a literature review tool to answer both high-level ("What is this paper about?") and specific questions ("What statistical test was used in Figure 3?"). The parent provides grounding, while children deliver citable, precise evidence.

A patient's visit summary is a parent chunk. Children are specific lab results, physician notes, medication lists, and imaging reports. A query about "last hemoglobin A1c" retrieves the lab result child. A query for "patient history" can retrieve the parent summary, or a synthesized view built by aggregating relevant children (all lab trends, all notes), providing a complete clinical picture.

A company policy document (e.g., "Remote Work Policy") is a parent. Its children are specific sections: Eligibility, Equipment Reimbursement, Tax Implications, Security Protocols. An employee asking "How do I get a monitor paid for?" gets the exact reimbursement child. An HR query for "What's in our remote work policy?" retrieves the parent, ensuring no critical section is omitted from the generated summary.

Systems implement this by storing two types of embeddings. Parent chunks are embedded for broad semantic search. Child chunks are embedded for detailed, fact-specific search. During retrieval, a hybrid strategy is used:

• Retrieve the top-K most relevant parents for context.
• Retrieve the top-N most relevant children for precise facts.
• The language model's context window is then populated with a combination of the best-matched parent and its most relevant children, optimizing for both scope and accuracy.

Hierarchical chunking is the overarching strategy that parent-child chunks implement. It creates a multi-level tree structure of text segments (e.g., document → chapter → section → paragraph). This enables multi-granular retrieval, where a query can be matched against summaries at a high level or detailed evidence at a low level. It is fundamental for navigating large, structured documents like legal contracts or technical manuals.

Semantic chunking splits text based on natural meaning boundaries like paragraphs, topics, or entities, rather than arbitrary character counts. It often serves as the first pass for creating intelligent parent chunks. The goal is to keep coherent ideas together, which improves the quality of embeddings for top-level retrieval before more granular child chunks are created within those semantic units.

Sentence window retrieval is a complementary RAG strategy focused on precision. A single, highly relevant sentence (analogous to a fine-grained child chunk) is retrieved via dense search. Its surrounding context (the "window") is then appended. This mirrors the parent-child philosophy: a precise anchor point (child) is enriched by its immediate context (parent-like window) for the final LLM prompt, balancing specificity with necessary background.

Recursive character text splitting is a widely used algorithmic approach to create chunks of a desired size. It recursively splits text using a hierarchy of separators (e.g., \n\n, \n, ., ). This method is frequently used as the underlying mechanism to generate child chunks within a larger parent chunk that was defined by a higher-level separator, ensuring child chunks respect sentence and word boundaries.

Chunk granularity refers to the level of detail in a text segment, from coarse (entire documents) to fine (single sentences). The parent-child pattern is a direct implementation of multi-granularity.

• Coarse-grained (Parent): Better for high-recall retrieval, capturing broad context.
• Fine-grained (Child): Better for high-precision retrieval, providing exact evidence. The choice directly trades off between retrieval recall and the relevance of the context provided to the LLM.