Pipeline

A LaTeX formula enters TeXForm as a string and leaves through one of four exits. This page is the map: it shows the single path every formula takes and links each stage to the concept page that explains it in depth.

What the pipeline is#

TeXForm processes a formula in one direction. The parser turns source text into a SyntaxNode, that snapshot becomes an editable Document, and the Document exits as serialized text, tokenized text, a normalized formula, or a structured snapshot.

The flow is single-direction: source becomes a snapshot, the snapshot becomes a working tree, and the working tree produces output. Nothing loops back on its own; you re-enter the pipeline by parsing a new string.

The parser produces a snapshot first#

The parser reads source text and consults a knowledge base of command and environment signatures to decide how each token binds its arguments. Its immediate output is a SyntaxNode: an immutable, lossless snapshot of the parse, which may contain Error nodes for fragments it could not parse. The snapshot is never edited in place.

See Parsing for why the parser needs a knowledge base and what the three possible parse results mean.

SyntaxNode is the wire format, Document is the working format#

The most useful distinction in TeXForm is between its two public trees:

• A SyntaxNode is the wire format — the snapshot you serialize to JSON, store, transport, or use as a test fixture. It carries no editing behavior.
• A Document is the working format — a DOM-style tree you query, mutate, serialize, and transform. Reads go through lightweight node handles; edits are validated and return a structured error instead of panicking.

TeXForm converts the snapshot into a Document for you on the parse path, and bridges back the other way on demand. The two directions are symmetric over every node kind, including Error and Prime.

import texform

doc = texform.Parser().parse(r"x^2")["document"]
snapshot = doc.to_syntax()                          # working format -> wire format
restored = texform.Document.from_syntax(snapshot)   # wire format -> working format
print(restored.to_latex())

x ^ { 2 }

A rebuilt document is bound to a knowledge base like any other: from_syntax uses the shared default unless you pass one. TransformEngine.transform accepts it when that is the engine's own knowledge-base instance and rejects it as foreign otherwise, so pass engine.knowledge_base() when you rebuild a document you intend to normalize in place.

Four exits#

A Document exits in four ways, each with a different purpose:

• to_latex() serializes the tree back to LaTeX text, spaced by default or compact on request. Even a tree containing Error nodes round-trips: each error re-emits its captured source snippet verbatim. See Serialization.
• to_tokenized_latex() runs that same serializer traversal with a recorder attached, returning the identical LaTeX string plus typed tokens with byte spans into it. It is one traversal, not a second lexing pass. See Tokenized output.
• TransformEngine normalizes the formula into a canonical form chosen by a profile. Normalization is gated on a complete tree — a tree with errors cannot be normalized. See Transforms.
• to_syntax() converts the tree back to a SyntaxNode for structured-data output. This is the only exit that produces serde-friendly structured data; there is no method named serialize on Document.