chunk.MarkdownChunk

A chunk extracted from a Markdown document.

Usage

Source

chunk.MarkdownChunk(
    text,
    start_index,
    end_index,
    char_count,
    context=None,
    origin=None,
    attributes=None
)

MarkdownChunker produces MarkdownChunks. It behaves exactly like a Chunk; the one thing to note is that its context field holds the Markdown heading hierarchy in effect at the chunk’s position (the #/## headings the passage falls under). That context lets retrieval results show where in a document a passage came from, which helps both with ranking and with citing sources back to the user.

Parameters

text: str

The text content of the chunk.

start_index: int

Character position where this chunk begins in the source document.

end_index: int

Character position where this chunk ends in the source document.

char_count: int

Number of characters in this chunk.

context: Optional[str] = None

The Markdown heading hierarchy in effect at this chunk’s position, or None if there is none.

origin: Optional[str] = None

Origin of the parent document this chunk belongs to.

attributes: Optional[dict[str, Any]] = None

Optional user-defined attributes associated with the chunk, used for retrieval filtering and downstream prompt construction.