Markdown/HTML Splitting

Markdown/HTML splitting uses the document's inherent Document Object Model (DOM) or markdown syntax tree to define chunk boundaries. Instead of arbitrary character counts, it splits at logical structural elements:

• Headings (<h1>, #)
• Paragraphs (<p>, blank lines)
• Lists (<ul>, -)
• Code blocks (<pre>, ```)
• Horizontal rules (<hr>, ---)
• Table rows (<tr>) This preserves the author's intended grouping of ideas, leading to chunks that are more self-contained and meaningful for retrieval.

The primary advantage is maintaining semantic coherence within each chunk. A chunk created by splitting at a heading (## Results) will contain all content related to that subsection until the next heading of equal or greater importance. This contrasts with fixed-length splitting, which can sever a sentence mid-thought or separate a data point from its explanatory context. Coherent chunks improve retrieval precision because the embedded vector representation more accurately reflects a complete concept or topic.

Markup languages are inherently hierarchical. Splitters can leverage this to create parent-child chunk relationships automatically. For example:

• Parent Chunk: A section defined by an <h2> heading.
• Child Chunks: Individual paragraphs or lists within that section. This enables multi-granularity retrieval. A broad query can retrieve the parent chunk for an overview, while a specific query can retrieve a precise child chunk. This structure is foundational for advanced retrieval patterns like sentence window retrieval where a core child chunk is expanded with its parent's context.

During splitting, valuable metadata can be extracted from the markup and attached to each chunk, enriching it for filtering and ranking. This includes:

• Heading level and text (for hierarchical context)
• HTML element type (e.g., table, code)
• CSS class names or IDs (e.g., class="important")
• Markdown formatting (e.g., bold text indicating key terms)
• Link destinations (href attributes) This metadata allows for hybrid retrieval strategies, combining semantic vector search with precise metadata filters (e.g., WHERE element_type = 'code') to quickly narrow results.

A key challenge in chunking is context fragmentation, where related information is split across separate chunks. Markdown/HTML splitting reduces this by using large structural blocks as primary units. To handle cases where these blocks exceed desired token limits, it is often combined with recursive splitting. The strategy is applied hierarchically: first split by major headings, then by paragraphs, then by sentences if needed. This ensures the largest semantically intact unit is always preserved first, minimizing information loss at boundaries.

This strategy is implemented in major RAG development frameworks:

• LangChain: The MarkdownHeaderTextSplitter and HTMLHeaderTextSplitter split based on headers and preserve header metadata.
• LlamaIndex: The HTMLNodeParser and MarkdownNodeParser convert elements/headers into Node objects with inherited metadata.
• Unstructured.io: The library's partitioning functions natively understand HTML/XML tags and Markdown syntax to output structured elements. These tools handle the complexity of nested tags and inconsistent markup, allowing engineers to focus on configuring chunk granularity (e.g., split by h2 only) rather than writing custom parsers.

Effective markdown/HTML splitting requires more than just applying a library. Key engineering considerations include:

• Combine with Recursive Splitting: Use header-based splitting first, then apply a recursive character splitter to long sections to respect the model's context window.
• Manage Chunk Overlap: Implement chunk overlap (e.g., 10% of chunk size) when splitting within a structural element to prevent information loss at seams.
• Validate Output: Post-splitting, analyze chunk length distributions and semantic coherence to tune parameters like chunk_size and chunk_overlap.

The quality of splitting directly impacts RAG system performance. Key evaluation metrics and considerations:

• Retrieval Recall: Structural splitting improves recall by keeping related concepts (e.g., a list and its introductory paragraph) together.
• Answer Precision: Well-formed chunks reduce the chance of the LLM receiving incomplete context, mitigating hallucinations.
• Benchmarking: Use retrieval evaluation metrics like Hit Rate or Mean Reciprocal Rank (MRR) to A/B test different splitting strategies (e.g., fixed-length vs. markdown-based) on your corpus.

A strategy that splits text based on its natural semantic boundaries, such as the conclusion of a topic, a shift in narrative, or the end of a coherent argument. Unlike fixed-length or delimiter-based methods, it aims to keep self-contained ideas within a single chunk, which can improve retrieval relevance.

• Primary Mechanism: Often uses a language model or a heuristic (like a significant drop in cosine similarity between sentences) to identify breakpoints.
• Advantage: Produces chunks with higher intrinsic coherence, which can lead to more precise semantic search results.
• Trade-off: More computationally expensive than rule-based methods and can be sensitive to the chosen model or threshold.

A hierarchical, rule-based strategy that recursively splits text using a prioritized list of separators (e.g., \n\n, \n, . , ) until the resulting chunks are within a desired size range.

• Process: It first attempts to split on the primary separator (e.g., double newlines for paragraphs). If chunks are still too large, it moves to the next separator (e.g., single newlines), and so on.
• Use Case: A robust, general-purpose method that works well on plain text where structural markup is not available. It's the default splitter in many frameworks like LangChain.
• Key Parameter: The chunk_size and chunk_overlap settings control the final output granularity.

A strategy for semi-structured documents (PDFs, scanned forms, HTML tables) that uses visual and spatial cues—not just textual ones—to define chunk boundaries. It is closely related to markdown/HTML splitting but applied to rendered document layouts.

• Input Sources: PDFs where text position, font size, and column boundaries convey structure.
• Techniques: Leverages libraries like PDFPlumber, Docling, or Unstructured.io to extract not just text, but also its bounding boxes and reading order.
• Output: Chunks that respect logical visual units, such as a figure with its caption, a table, or a sidebar, preserving information that pure text splitting would scramble.

A strategy that creates a multi-level tree structure of chunks (e.g., document > section > subsection > paragraph) to enable retrieval at different levels of granularity. Markdown/HTML, with its inherent heading hierarchy, is a natural input for this method.

• Parent-Child Relationships: A large 'parent' chunk (e.g., a whole section) contains smaller 'child' chunks (e.g., individual paragraphs within it).
• Retrieval Flexibility: A broad query can retrieve a parent chunk for overview context, while a specific query can retrieve a precise child chunk for a detailed answer.
• Implementation: Often stored in vector databases that support hierarchical or multi-vector indexing, allowing the retrieval system to choose the appropriate level.

A specialized strategy for source code that parses code into its syntactic tree structure and uses language-defined nodes (functions, classes, methods) as logical, self-contained chunks. It is the semantic equivalent of markdown/HTML splitting for programming languages.

• Process: A parser (like tree-sitter) builds an AST where each node represents a syntactic element. Chunks are created by extracting the text of subtrees corresponding to logical units.
• Benefit: Preserves code semantics perfectly; a chunk is a complete, compilable function or class, not an arbitrary slice of text that might break syntax.
• Application: Essential for code retrieval-augmented generation (RAG) systems, where retrieving a half-function is useless.

A retrieval strategy (not strictly a chunking strategy) that uses fine-grained chunks but expands context at query time. It often relies on high-quality sentence splitting as a prerequisite, similar to using <p> tags in HTML.

• Two-Stage Process:
  1. Indexing: Individual sentences are embedded and indexed as separate, fine-grained chunks.
  2. Retrieval & Expansion: When a sentence is retrieved, its immediate surrounding sentences (a 'window') are also fetched to provide the language model with necessary context.
• Advantage: Combines the precision of retrieving a single relevant sentence with the contextual completeness of a larger passage, optimizing for both accuracy and information sufficiency.