`) and inline HTML (like ``). It also allows potentially dangerous URL schemes like `javascript:` in links. Only enable this for content you fully trust. ## What Comes Next This page covered the basics of HTML rendering with multimark. The following pages explore GFM extensions, the other output formats, and the options system in depth. Each builds on what you have seen here, using the same straightforward API pattern: pass text in, get rendered output back. ## Core Concepts ### GFM Extensions This page explains the five GitHub Flavored Markdown extensions that multimark supports, with examples showing how each one works and how to combine them. ## What Are GFM Extensions? GitHub Flavored Markdown (GFM) extends the CommonMark specification with five additional syntax features. These are the constructs that GitHub recognizes in README files, issues, and pull requests but that are not part of the base CommonMark spec. Multimark supports all five GFM extensions through the `extensions` parameter, which accepts a sequence of extension names. Extensions are disabled by default so that the parser remains strictly CommonMark-compliant unless you opt in. ```{python} from multimark import VALID_EXTENSIONS sorted(VALID_EXTENSIONS) ``` Each extension is independent: you enable only the ones you need. ## Tables The `table` extension adds pipe-table syntax. Columns are separated by `|` characters, and a header row is separated from body rows by a line of dashes. ```{python} from multimark import markdown_to_html table_md = """\ | Feature | Status | |------------|--------| | Tables | ✓ | | Autolinks | ✓ | | Tasklists | ✓ | """ print(markdown_to_html(table_md, extensions=["table"])) ``` Column alignment is controlled with colons in the separator row: `:---` for left, `:---:` for center, and `---:` for right. The renderer emits `style="text-align: ..."` attributes on the `` and `` elements. ```{python} aligned = """\ | Left | Center | Right | |:-----|:------:|------:| | a | b | c | """ print(markdown_to_html(aligned, extensions=["table"])) ``` ## Strikethrough The `strikethrough` extension recognizes `~~text~~` as deleted content, rendering it with `` tags. ```{python} print(markdown_to_html("This is ~~no longer~~ relevant.", extensions=["strikethrough"])) ``` Strikethrough works inline alongside other formatting. You can combine it with bold, italic, and code spans without issue. ```{python} combined = "~~**bold and struck**~~ and ~~`code struck`~~" print(markdown_to_html(combined, extensions=["strikethrough"])) ``` ## Autolinks The `autolink` extension automatically converts bare URLs and email addresses into clickable links without requiring explicit Markdown link syntax. ```{python} text = """\ Visit https://github.com for more info. Send feedback to user@example.com. """ print(markdown_to_html(text, extensions=["autolink"])) ``` This is the same behavior you see on GitHub: any URL starting with `http://` or `https://` becomes a link, and email addresses get `mailto:` links. ## Tag Filtering The `tagfilter` extension suppresses a specific set of HTML tags that could be used for XSS attacks, even when `unsafe=True` is set. The filtered tags are: ``, `<style>`, `<xmp>`, `<iframe>`, `<noembed>`, `<noframes>`, `<script>`, and `<plaintext>`. ```{python} html_input = "Before <script>alert('xss')</script> after" print(markdown_to_html(html_input, unsafe=True, extensions=["tagfilter"])) ``` The tags are not removed entirely. Instead, the opening `<` is replaced with `<`, rendering them inert while preserving the text content for visibility. This matches GitHub's behavior for user-generated content. ## Task Lists The `tasklist` extension renders list items that begin with `[ ]` or `[x]` as checkbox items with a special CSS class. ```{python} tasks = """\ - [x] Write documentation - [x] Add tests - [ ] Publish release """ print(markdown_to_html(tasks, extensions=["tasklist"])) ``` The checkboxes are rendered as `<input>` elements with `type="checkbox"` and a `disabled` attribute (since they are static content, not interactive form elements). ## Combining Extensions Extensions compose freely. Enable as many as you need by passing multiple names in the `extensions` list. ```{python} doc = """\ ## Release Notes | Change | Status | |---------------|--------------| | ~~Old API~~ | Removed | | New API | Added | Tasks remaining: - [x] Update docs at https://example.com/docs - [ ] Tag release Contact: releases@example.com """ print(markdown_to_html(doc, extensions=["table", "strikethrough", "tasklist", "autolink"])) ``` The order of extension names in the list does not matter. The parser enables all requested extensions before processing the input. ## Extensions with Other Renderers GFM extensions work with all five output formats, not just HTML. Here is the same table rendered as LaTeX. ```{python} from multimark import markdown_to_latex print(markdown_to_latex(table_md, extensions=["table"])) ``` The LaTeX renderer maps table structure to `tabular` environments. Each renderer translates extension syntax into its native equivalent where possible. ### Output Formats This page explores the five output formats that multimark can produce from a single Markdown input, showing what each format looks like and when to reach for it. ## Five Renderers, One Parser All five output renderers share the same underlying parser. The Markdown is parsed once into an abstract syntax tree (AST), and then a format-specific renderer walks the tree to produce output. This means that parsing behavior (extensions, options) is consistent regardless of which renderer you choose. The renderers are: - `markdown_to_html()` for web content - `markdown_to_latex()` for typesetting - `markdown_to_man()` for Unix manual pages - `markdown_to_xml()` for AST inspection - `markdown_to_commonmark()` for normalized Markdown Each is a standalone function that you call with a Markdown string and optional parameters. ## HTML HTML is the most common output format. It produces well-formed fragments suitable for embedding in web pages, documentation systems, or email templates. ```{python} from multimark import markdown_to_html doc = """\ ## Features Multimark is **fast** and supports: 1. CommonMark 2. GFM extensions 3. Multiple output formats """ print(markdown_to_html(doc)) ``` The HTML renderer supports one parameter that the others do not: `sourcepos`. When enabled, each block-level element receives a `data-sourcepos` attribute indicating its line and column range in the original Markdown source. This is useful for building editors or tools that map rendered output back to source positions. ```{python} print(markdown_to_html("# Title\n\nParagraph\n", sourcepos=True)) ``` ## LaTeX The LaTeX renderer produces fragments suitable for inclusion in a `.tex` document. Headings map to sectioning commands, inline formatting maps to text commands, and code blocks use `verbatim` environments. ```{python} from multimark import markdown_to_latex doc = """\ # Introduction This is **bold** and *italic* with `inline code`. > A block quote. """ print(markdown_to_latex(doc)) ``` ### Line Wrapping The `width` parameter controls line wrapping in the LaTeX output. A value of `0` (the default) disables wrapping entirely. Any positive value wraps lines at approximately that column. ```{python} long_para = "This is a sentence that demonstrates line wrapping. " * 5 print(markdown_to_latex(long_para, width=60)) ``` Wrapping makes the LaTeX source more readable and produces cleaner diffs in version control. ## Man Pages The groff man page renderer produces output compatible with the `man` command and related roff-based toolchains. Formatting uses font-switching requests and standard man macros. ```{python} from multimark import markdown_to_man doc = """\ ## Synopsis **multimark** - Python bindings for cmark-gfm ## Description Converts *Markdown* to multiple output formats. """ print(markdown_to_man(doc)) ``` Like LaTeX, the man renderer supports `width` for line wrapping. Setting this to 72 or 78 matches common man page formatting conventions. ## XML The XML renderer produces a structured representation of the parsed AST. Every Markdown construct becomes a named XML element, making this format ideal for debugging, testing, or feeding into XML-based toolchains. ```{python} from multimark import markdown_to_xml simple = "A **bold** word.\n" print(markdown_to_xml(simple)) ``` The output includes an XML declaration and DOCTYPE reference. Each element directly corresponds to an AST node type: `<paragraph>`, `<strong>`, `<text>`, `<softbreak>`, and so on. ### Inspecting Document Structure The XML format is particularly valuable when you need to understand exactly how the parser interprets a given input. Ambiguous constructs or edge cases become immediately clear in the AST representation. ```{python} # How does the parser handle nested emphasis? tricky = "***bold and italic***\n" print(markdown_to_xml(tricky)) ``` ## CommonMark (Normalization) The CommonMark renderer parses Markdown and re-renders it in a canonical form. Different but semantically equivalent inputs produce identical output after normalization. ```{python} from multimark import markdown_to_commonmark # These three inputs all mean the same thing inputs = [ "_hello_ **world**\n", "*hello* __world__\n", "*hello* **world**\n", ] for md in inputs: print(repr(markdown_to_commonmark(md))) ``` All three inputs normalize to the same canonical form. This is useful for enforcing consistent formatting across a document or codebase, and for testing that transformations preserve semantics. ### Round-Trip Stability A key property of the CommonMark renderer is round-trip stability: normalizing an already-normalized document produces the same output. This means you can apply it idempotently. ```{python} original = "- First\n- Second\n- Third\n" once = markdown_to_commonmark(original) twice = markdown_to_commonmark(once) print(f"Stable: {once == twice}") print(once) ``` ## Choosing a Format The right renderer depends on your downstream pipeline. Here is a quick decision guide. ::: {.details .open summary="When to use each format" icon="compass"} **HTML** when you are producing web content, documentation, or any output that will be displayed in a browser. **LaTeX** when you are generating input for `pdflatex`, `xelatex`, or any TeX-based typesetting system. **Man** when you are writing Unix manual pages for command-line tools. **XML** when you need to inspect the AST, run XPath queries, or feed structured data into an XML pipeline. **CommonMark** when you want to normalize formatting, enforce style consistency, or verify round-trip stability of your Markdown processing pipeline. ::: ### Options and Flags This page explains how to control parsing and rendering behavior through keyword arguments, the `Options` bitmask, or a combination of both. ## Two Ways to Configure Every renderer accepts configuration through two complementary mechanisms: boolean keyword arguments and an `Options` bitmask. You can use either one or both together. They are merged with a bitwise OR, so there is no conflict between them. The keyword arguments cover the most commonly used flags: ```{python} from multimark import markdown_to_html # Using keyword arguments markdown_to_html("'Hello' -- world...", smart=True) ``` The `Options` enum covers every flag, including GFM-specific ones that have no keyword shortcut: ```{python} from multimark import markdown_to_html, Options # Using the Options bitmask markdown_to_html("'Hello' -- world...", options=Options.SMART) ``` Both calls produce the same result. The keyword style is more readable for one or two flags; the bitmask style is more concise when combining many flags into a reusable configuration. ## Available Keyword Arguments Every renderer accepts these boolean keywords (all default to `False`): | Keyword | Effect | |---------|--------| | `hardbreaks` | Render soft line breaks as `<br />` (HTML) or equivalent | | `smart` | Convert ASCII quotes and dashes to typographic characters | | `normalize` | Consolidate adjacent text nodes in the AST | | `unsafe` | Allow raw HTML and dangerous URLs to pass through | | `footnotes` | Enable footnote syntax (`[^label]`) | Additionally, `markdown_to_html` and `markdown_to_xml` accept `sourcepos` (include source position data in the output), while `markdown_to_latex`, `markdown_to_man`, and `markdown_to_commonmark` accept `width` (line wrapping column, where `0` means no wrapping). ## The Options Enum The `Options` class is an `IntFlag` enumeration. Each member corresponds to a cmark-gfm option bit. Because it is an `IntFlag`, members can be combined with the `|` operator. ```{python} from multimark import Options # Combine multiple flags opts = Options.SMART | Options.UNSAFE | Options.FOOTNOTES print(f"Combined value: {opts}") print(f"Individual flags: {opts!r}") ``` ### All Available Flags The full set of flags includes several GFM-specific options that have no keyword argument equivalent. ::: {.details summary="Complete Options reference" icon="list"} | Flag | Keyword Equivalent | Description | |------|-------------------|-------------| | `DEFAULT` | (none needed) | No flags set (value `0`) | | `SOURCEPOS` | `sourcepos=True` | Include source position attributes | | `HARDBREAKS` | `hardbreaks=True` | Render soft breaks as hard breaks | | `UNSAFE` | `unsafe=True` | Allow raw HTML passthrough | | `NOBREAKS` | (none) | Render soft breaks as spaces | | `NORMALIZE` | `normalize=True` | Consolidate adjacent text nodes | | `VALIDATE_UTF8` | (none) | Replace invalid UTF-8 sequences | | `SMART` | `smart=True` | Typographic quote/dash conversion | | `GITHUB_PRE_LANG` | (none) | Use `lang` attribute on `<pre>` instead of `class` on `<code>` | | `LIBERAL_HTML_TAG` | (none) | Be more permissive with HTML tag parsing | | `FOOTNOTES` | `footnotes=True` | Enable footnote syntax | | `STRIKETHROUGH_DOUBLE_TILDE` | (none) | Require `~~` for strikethrough (not `~`) | | `FULL_INFO_STRING` | (none) | Include full info string in code block attributes | ::: ## Combining Keywords and Bitmask When you pass both keyword arguments and an `options` bitmask, they are merged together. This lets you use keywords for common flags and the bitmask for GFM-specific ones. ```{python} from multimark import markdown_to_html, Options # smart=True sets SMART, options adds GITHUB_PRE_LANG fence = "```" code_block = f"'Hello' -- world\n\n{fence}python\nprint('hi')\n{fence}\n" print(markdown_to_html( code_block, smart=True, options=Options.GITHUB_PRE_LANG, )) ``` Notice how the code block uses `<pre lang="python">` instead of the default `<code class="language-python">`. The `GITHUB_PRE_LANG` flag changed the attribute style, while `smart=True` handled the punctuation. ## Reusable Configurations For applications that render many documents with the same settings, define your options once and reuse them. ```{python} from multimark import markdown_to_html, Options # A configuration for rendering user-generated content safely SAFE_GFM = Options.SMART | Options.VALIDATE_UTF8 | Options.STRIKETHROUGH_DOUBLE_TILDE def render_comment(text: str) -> str: """Render a user comment with safe defaults.""" return markdown_to_html( text, extensions=["table", "strikethrough", "autolink", "tasklist"], options=SAFE_GFM, ) render_comment("'Hello' -- check out https://example.com") ``` This pattern keeps rendering configuration centralized and easy to audit. If you later need to adjust the options, you change one definition rather than updating every call site. ## The SAFE vs. UNSAFE Distinction The `SAFE` and `UNSAFE` flags are complementary. `SAFE` is the legacy name for the default behavior (strip raw HTML). `UNSAFE` explicitly opts in to allowing raw HTML. ```{python} from multimark import markdown_to_html raw_html = "<marquee>Dangerous!</marquee>" # Default: raw HTML is stripped print("Safe:", markdown_to_html(raw_html)) # With unsafe=True: raw HTML passes through print("Unsafe:", markdown_to_html(raw_html, unsafe=True)) ``` When building applications that accept Markdown from untrusted sources, always leave `unsafe` at its default value of `False`. The cost of stripping raw HTML is negligible compared to the security risk of XSS vulnerabilities. ### Command-Line Interface Multimark includes a command-line tool for quick Markdown conversions without writing Python code. It reads from a file or stdin and writes the rendered output to stdout or a file. ## Basic Usage The simplest invocation converts Markdown to HTML: ```bash echo '# Hello **world**' | multimark ``` ```html <h1>Hello <strong>world</strong></h1> ``` You can also pass a filename directly: ```bash multimark README.md ``` ## Choosing an Output Format Use the `-t` / `--to` flag to select the output format. The available formats are `html` (the default), `latex`, `man`, `commonmark`, and `xml`. ```bash echo '# Hello' | multimark --to latex ``` ```latex \section{Hello} ``` ```bash echo '**bold** and *italic*' | multimark --to xml ``` ```xml <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE document SYSTEM "CommonMark.dtd"> <document xmlns="http://commonmark.org/xml/1.0"> <paragraph> <strong> <text xml:space="preserve">bold</text> </strong> <text xml:space="preserve"> and </text> <emph> <text xml:space="preserve">italic</text> </emph> </paragraph> </document> ``` ## Writing to a File Use `-o` / `--output` to write results to a file instead of stdout: ```bash multimark README.md --to html -o README.html ``` ## Enabling GFM Extensions GFM extensions are enabled with the `-e` / `--extension` flag. It can be repeated to enable multiple extensions at once. ```bash echo '| A | B |\n|---|---|\n| 1 | 2 |' | multimark -e table ``` The available extensions are `autolink`, `strikethrough`, `table`, `tagfilter`, and `tasklist`. ```bash multimark doc.md -e table -e strikethrough -e autolink ``` ## Rendering Options Several flags control how the Markdown is parsed and rendered: | Flag | Effect | |------|--------| | `--smart` | Convert straight quotes to curly, `--` to en-dash, `---` to em-dash, `...` to ellipsis | | `--unsafe` | Allow raw HTML and potentially dangerous URLs to pass through | | `--hardbreaks` | Render soft line breaks as hard breaks | | `--sourcepos` | Add source position attributes (html and xml only) | | `--footnotes` | Enable footnote syntax | These can be combined freely: ```bash echo '"Hello" -- world...' | multimark --smart ``` ```html <p>\u201cHello\u201d \u2013 world\u2026</p> ``` ```bash echo '<div>raw</div>' | multimark --unsafe ``` ```html <div>raw</div> ``` ## Reading from stdin When no file argument is provided, multimark reads from stdin. This makes it easy to use in shell pipelines: ```bash curl -s https://example.com/README.md | multimark --to html > output.html ``` ```bash cat doc.md | multimark --smart --to latex | tee output.tex ``` ## Version Check the installed version with: ```bash multimark --version ``` ## Full Help ```bash multimark --help ``` ``` Usage: multimark [OPTIONS] [FILE] Convert CommonMark/GFM Markdown to various output formats. Reads Markdown from FILE (or stdin if omitted) and writes the converted output to stdout or the file specified by --output. Options: -t, --to [html|latex|man|commonmark|xml] Output format. -o, --output FILENAME Output file (stdout if omitted). -e, --extension [autolink|strikethrough|table|tagfilter|tasklist] Enable a GFM extension (repeatable). --smart Use smart punctuation. --unsafe Allow raw HTML and dangerous URLs. --hardbreaks Render softbreaks as hard line breaks. --sourcepos Include source position attributes (html/xml only). --footnotes Enable footnote parsing. --version Show the version and exit. --help Show this message and exit. ``` ## Advanced Usage ### Security Considerations This page covers the security model that multimark uses when rendering untrusted Markdown, and how to relax it safely when you need raw HTML passthrough. ## The Default Is Safe Multimark strips raw HTML from input by default. This means you can safely pass untrusted Markdown (user comments, forum posts, uploaded documents) through any renderer without risking cross-site scripting (XSS) or HTML injection. ```{python} from multimark import markdown_to_html # Untrusted input with malicious HTML untrusted = """ Check this out: <script>document.cookie</script> <img src=x onerror="alert('xss')"> """ print(markdown_to_html(untrusted)) ``` Both the `<script>` tag and the malicious `<img>` are replaced with harmless comments. The text content of the document is preserved; only the raw HTML is neutralized. ## What Gets Stripped When `unsafe=False` (the default), the following are all replaced with ``: - Block-level HTML (e.g., `<div>`, `<table>`, `<script>`) - Inline HTML (e.g., `<span>`, `<img>`, `<a>` tags written directly) - Dangerous URL schemes (e.g., `javascript:`, `vbscript:`, `data:` in links) Standard Markdown link and image syntax is not affected. Only literal HTML tags embedded in the Markdown source are stripped. ```{python} # Markdown links work fine (they produce safe, well-formed HTML) print(markdown_to_html("[Click here](https://example.com)")) # But raw <a> tags are stripped print(markdown_to_html('<a href="javascript:alert(1)">Click</a>')) ``` ## When to Use `unsafe=True` There are legitimate cases where you need raw HTML passthrough: - You control the Markdown source entirely (e.g., your own documentation) - You are building a static site generator where authors are trusted - You need to embed custom HTML components (videos, widgets, forms) In these situations, `unsafe=True` is appropriate because the input is not adversarial. ```{python} # Trusted content: embedding a video player trusted = """\ ## Demo Video <video controls width="640"> <source src="demo.mp4" type="video/mp4"> </video> The video above shows the installation process. """ print(markdown_to_html(trusted, unsafe=True)) ``` ## Tag Filtering as a Middle Ground The `tagfilter` GFM extension provides a middle ground between full stripping and full passthrough. It allows most HTML through but neutralizes a specific set of dangerous tags: `<textarea>`, `<style>`, `<xmp>`, `<iframe>`, `<noembed>`, `<noframes>`, `<script>`, and `<plaintext>`. ```{python} # tagfilter + unsafe: allow safe HTML, block dangerous tags mixed = """\ <div class="note">This is fine</div> <script>alert('blocked')</script> <em>Also fine</em> """ print(markdown_to_html(mixed, unsafe=True, extensions=["tagfilter"])) ``` The dangerous tags have their opening `<` replaced with `<`, rendering them visible as text rather than executable HTML. Other tags like `<div>` and `<em>` pass through normally. This matches GitHub's approach to rendering user content. ## Defense in Depth For applications processing user-generated content, multimark's default safe mode is your first line of defense. However, good security practice involves multiple layers. ::: {.details summary="Recommended security layers" type="tip" icon="shield"} 1. **Keep `unsafe=False`** (the default). This eliminates the vast majority of XSS vectors at the Markdown-to-HTML conversion step. 2. **Use `tagfilter`** if you need some HTML passthrough. It blocks the most dangerous tags while allowing benign formatting. 3. **Apply a secondary HTML sanitizer** (like [Bleach](https://github.com/mozilla/bleach) or [nh3](https://github.com/messense/nh3)) after rendering if your threat model requires belt-and-suspenders protection. 4. **Set Content-Security-Policy headers** on your web server to prevent inline script execution regardless of what HTML reaches the browser. ::: These layers work together so that a failure in any one layer does not expose your users to an attack. ### Practical Recipes This page collects practical patterns for common tasks: reading files, building converters, generating LaTeX and man pages, batch processing, and integrating with template engines. ## Rendering Files from Disk Real-world usage often involves reading Markdown from files rather than inline strings. The pattern is straightforward: read the file, render it, use the output. ```{python} #| eval: false from pathlib import Path from multimark import markdown_to_html content = Path("README.md").read_text(encoding="utf-8") html = markdown_to_html(content, extensions=["table", "autolink"]) ``` Always specify `encoding="utf-8"` when reading files. Markdown files should be UTF-8 encoded, and multimark expects `str` input (not bytes). ## Building a Simple Markdown-to-HTML Converter A complete file converter needs just a few lines. This example reads a Markdown file, wraps the rendered HTML in a minimal document structure, and writes the result. ```{python} #| eval: false from pathlib import Path from multimark import markdown_to_html def convert_file(src: Path, dst: Path) -> None: """Convert a Markdown file to a standalone HTML file.""" md = src.read_text(encoding="utf-8") body = markdown_to_html(md, smart=True, extensions=["table", "autolink"]) html = f"""\ <!DOCTYPE html> <html lang="en"> <head><meta charset="utf-8"><title>{src.stem}</title></head> <body> {body} </body> </html>""" dst.write_text(html, encoding="utf-8") ``` This gives you a functional static site generator in under 20 lines. ## Generating LaTeX for Academic Papers When writing academic papers, you might author content in Markdown and convert it to LaTeX for inclusion in a larger document. The `width` parameter keeps the LaTeX source readable. ```{python} from multimark import markdown_to_latex abstract = """\ ## Abstract We present a novel approach to Markdown processing that combines the speed of compiled C with the convenience of Python. Our benchmarks show a 50x improvement over pure-Python parsers while maintaining full CommonMark compliance. ## Keywords - Markdown - CommonMark - Natural language processing """ print(markdown_to_latex(abstract, smart=True, width=72)) ``` The output can be pasted directly into a LaTeX document between `\begin{document}` and `\end{document}`. The sectioning commands (`\section`, `\subsection`) integrate naturally with LaTeX's document structure. ## Creating Man Pages for CLI Tools If you maintain command-line tools, you can author the documentation in Markdown and generate the groff source for man pages. ```{python} from multimark import markdown_to_man manpage = """\ ## Name **mytools** - a collection of useful utilities ## Synopsis **mytools** *command* [*options*] ## Description Mytools provides several commands for everyday development tasks. Use **mytools help** to see a list of available commands. ## Options - **--verbose**, **-v**: Enable verbose output - **--quiet**, **-q**: Suppress all output except errors """ print(markdown_to_man(manpage, width=72)) ``` Wrap this output in a `.TH` header and save it as `mytools.1` to produce a complete man page that you can install with your package. ## Footnotes The `footnotes` option enables footnote syntax, which is useful for academic writing, technical documentation, and blog posts. ```{python} from multimark import markdown_to_html doc = """\ Markdown was created by John Gruber[^1] with contributions from Aaron Swartz[^2]. [^1]: John Gruber is the author of Daring Fireball. [^2]: Aaron Swartz was a programmer and internet activist. """ print(markdown_to_html(doc, footnotes=True)) ``` Footnote references become superscript links, and the footnote content is collected at the bottom of the document with back-links to the reference site. ## Batch Processing When converting many files, the overhead of Python function calls dominates rather than parsing time. The parser initializes quickly, so processing files in a loop is efficient. ```{python} #| eval: false from pathlib import Path from multimark import markdown_to_html source_dir = Path("docs/") output_dir = Path("site/") output_dir.mkdir(exist_ok=True) extensions = ["table", "strikethrough", "autolink", "tasklist"] for md_file in sorted(source_dir.glob("**/*.md")): html = markdown_to_html( md_file.read_text(encoding="utf-8"), smart=True, extensions=extensions, ) out_path = output_dir / md_file.relative_to(source_dir).with_suffix(".html") out_path.parent.mkdir(parents=True, exist_ok=True) out_path.write_text(html, encoding="utf-8") ``` This pattern scales to thousands of files without issue. The C parser handles each document in microseconds, so the bottleneck is typically disk I/O rather than rendering. ## Normalizing Markdown in a Pre-commit Hook The CommonMark renderer can enforce consistent formatting across a repository. Use it in a pre-commit hook to normalize Markdown files before they are committed. ```{python} #| eval: false from pathlib import Path from multimark import markdown_to_commonmark def normalize_markdown(path: Path) -> bool: """Normalize a Markdown file in place. Returns True if changed.""" original = path.read_text(encoding="utf-8") normalized = markdown_to_commonmark(original, width=80) if original != normalized: path.write_text(normalized, encoding="utf-8") return True return False ``` Setting `width=80` produces line-wrapped output that works well with `git diff`. Changes to content show up clearly in diffs rather than being obscured by reflowed paragraphs. ## Combining with Jinja2 Templates For documentation systems or email generators, combine multimark with a template engine to produce complete HTML pages with consistent styling. ```{python} #| eval: false from jinja2 import Template from multimark import markdown_to_html template = Template("""\ <!DOCTYPE html> <html> <head><title>{{ title }}</title></head> <body> <nav>{{ nav }}</nav> <main>{{ content }}</main> </body> </html> """) content_md = "# Hello\n\nWelcome to the site.\n" page = template.render( title="Home", nav="<a href='/'>Home</a>", content=markdown_to_html(content_md, smart=True), ) ``` This separation of concerns keeps your Markdown content clean while the template handles layout, navigation, and styling. ### Performance This page benchmarks multimark against other popular Python Markdown parsers and explains why multimark is fast. ## Benchmark: Markdown to HTML We render a ~4 KB Markdown document (with headings, lists, code blocks, inline formatting, and a table) **1,000 times** and measure the average time per call. All parsers produce HTML output. The benchmark document is repeated 10 times to reach a realistic size, and each library is configured with GFM table and strikethrough support where available. ```{python} #| echo: false import timeit # Build a realistic benchmark document fence = "```" doc = ( "# Benchmark Document\n\n" "This is a paragraph with **bold**, *italic*, and `code` formatting.\n\n" "## Lists\n\n" "- First item with a [link](https://example.com)\n" "- Second item with ~~strikethrough~~\n" "- Third item\n\n" "## Code\n\n" f"{fence}python\ndef hello():\n return \"world\"\n{fence}\n\n" "## Table\n\n" "| Column A | Column B | Column C |\n" "|----------|----------|----------|\n" "| 1 | 2 | 3 |\n" "| 4 | 5 | 6 |\n\n" "> A block quote to round things out.\n\n" ) * 10 # ~4 KB document iterations = 1000 # --- multimark --- from multimark import markdown_to_html t_multimark = timeit.timeit( lambda: markdown_to_html(doc, extensions=["table", "strikethrough"]), number=iterations, ) / iterations * 1_000_000 # µs # --- cmarkgfm --- import cmarkgfm t_cmarkgfm = timeit.timeit( lambda: cmarkgfm.github_flavored_markdown_to_html(doc), number=iterations, ) / iterations * 1_000_000 # --- mistune --- import mistune renderer_mistune = mistune.create_markdown() t_mistune = timeit.timeit( lambda: renderer_mistune(doc), number=iterations, ) / iterations * 1_000_000 # --- markdown-it-py --- from markdown_it import MarkdownIt md_it = MarkdownIt("gfm-like") t_markdown_it = timeit.timeit( lambda: md_it.render(doc), number=iterations, ) / iterations * 1_000_000 # --- commonmark.py --- import commonmark t_commonmark = timeit.timeit( lambda: commonmark.commonmark(doc), number=iterations, ) / iterations * 1_000_000 ``` ```{python} #| echo: false from great_tables import GT, style, loc import polars as pl results = pl.DataFrame({ "Package": ["multimark", "cmarkgfm", "mistune", "markdown-it-py", "commonmark.py"], "Engine": ["cmark-gfm (C)", "cmark-gfm (C)", "Pure Python", "Pure Python", "Pure Python"], "Time (µs)": [t_multimark, t_cmarkgfm, t_mistune, t_markdown_it, t_commonmark], }).with_columns( pl.col("Time (µs)").round(1), (pl.col("Time (µs)") / t_multimark).round(1).alias("Relative"), ).sort("Time (µs)") ( GT(results) .tab_header( title="Markdown → HTML Benchmark", subtitle=f"{len(doc):,}-byte document, {iterations:,} iterations, best-of-average", ) .fmt_number("Time (µs)", decimals=1) .fmt_number("Relative", decimals=1, pattern="{x}×") .tab_source_note("Lower is better. 'Relative' is normalized to multimark = 1.0×.") .tab_style( style=style.text(weight="bold"), locations=loc.body(rows=[0]), ) .tab_options( table_width="100%", quarto_disable_processing=True ) ) ``` ## Why multimark Is Fast multimark wraps a compiled C library via CFFI. The actual parsing and rendering happens entirely in C, with Python involved only for the function call overhead and string encoding/decoding. This architecture gives you the ergonomics of Python with the speed of native code. For typical Markdown documents (a few kilobytes), rendering completes in microseconds. The overhead of the Python-to-C boundary is negligible compared to what a pure-Python parser would spend on the same input. ## Memory Efficiency Each render call allocates memory for the AST and output buffer in C, then frees it before returning. There is no persistent state between calls, which means memory usage stays constant regardless of how many documents you process. ```{python} import sys from multimark import markdown_to_html result = markdown_to_html("# Hello\n\nWorld\n") print(f"Result type: {type(result).__name__}") print(f"Result size: {sys.getsizeof(result)} bytes") ``` The C library manages its own memory internally. Python only holds the final output string, so you do not need to worry about manual memory management or cleanup. ## Tips for Maximum Throughput ::: {.details summary="Performance tips" type="tip" icon="zap"} **Minimize extension overhead.** Each enabled extension adds a small amount of parsing work. If you do not need tables or autolinks for a particular batch, omit them. **Reuse option values.** Building the options bitmask is cheap, but defining it once outside a loop avoids redundant work. **Process in batches, not one-at-a-time.** If you are reading files from disk, batch the I/O and render calls together rather than interleaving with other heavy operations. **Avoid unnecessary encoding.** multimark accepts `str` and handles UTF-8 encoding internally. Do not pre-encode to bytes and decode back; just pass the string directly. :::

` through `