# Pipeline

How a LaTeX formula travels through TeXForm, from source text through the parser and the document tree to its four exits.

https://texform.dev/docs/pipeline

A LaTeX formula enters TeXForm as a string and leaves through one of four exits. This page is the map: it shows the single path every formula takes and links each stage to the concept page that explains it in depth.

## What the pipeline is [#what-the-pipeline-is]

TeXForm processes a formula in one direction. The parser turns source text into a `SyntaxNode`, that snapshot becomes an editable `Document`, and the `Document` exits as serialized text, tokenized text, a normalized formula, or a structured snapshot.

LaTeX source → Parser and knowledge base → SyntaxNode snapshot → editable Document. The Document supports text output, typed tokens, normalization, and structured snapshots.

The flow is single-direction: source becomes a snapshot, the snapshot becomes a working tree, and the working tree produces output. Nothing loops back on its own; you re-enter the pipeline by parsing a new string.

## The parser produces a snapshot first [#the-parser-produces-a-snapshot-first]

The parser reads source text and consults a knowledge base of command and environment signatures to decide how each token binds its arguments. Its immediate output is a `SyntaxNode`: an immutable, lossless snapshot of the parse, which may contain `Error` nodes for fragments it could not parse. The snapshot is never edited in place.

See [Parsing](https://texform.dev/docs/parsing) for why the parser needs a knowledge base and what the three possible parse results mean.

## SyntaxNode is the wire format, Document is the working format [#syntaxnode-is-the-wire-format-document-is-the-working-format]

The most useful distinction in TeXForm is between its two public trees:

* A `SyntaxNode` is the **wire format** — the snapshot you serialize to JSON, store, transport, or use as a test fixture. It carries no editing behavior.
* A `Document` is the **working format** — a DOM-style tree you query, mutate, serialize, and transform. Reads go through lightweight node handles; edits are validated and return a structured error instead of panicking.

TeXForm converts the snapshot into a `Document` for you on the parse path, and bridges back the other way on demand. The two directions are symmetric over every node kind, including `Error` and `Prime`.

```python
import texform

doc = texform.Parser().parse(r"x^2")["document"]
snapshot = doc.to_syntax()                          # working format -> wire format
restored = texform.Document.from_syntax(snapshot)   # wire format -> working format
print(restored.to_latex())
```

```text
x ^ { 2 }
```

A rebuilt document is bound to a knowledge base like any other: `from_syntax` uses the shared default unless you pass one. `TransformEngine.transform` accepts it when that is the engine's own knowledge-base instance and rejects it as foreign otherwise, so pass `engine.knowledge_base()` when you rebuild a document you intend to normalize in place.

## Four exits [#four-exits]

A `Document` exits in four ways, each with a different purpose:

* **`to_latex()`** serializes the tree back to LaTeX text, spaced by default or compact on request. Even a tree containing `Error` nodes round-trips: each error re-emits its captured source snippet verbatim. See [Serialization](https://texform.dev/docs/serialization).
* **`to_tokenized_latex()`** runs that same serializer traversal with a recorder attached, returning the identical LaTeX string plus typed tokens with byte spans into it. It is one traversal, not a second lexing pass. See [Tokenized output](https://texform.dev/docs/serialization#tokenized-output).
* **`TransformEngine`** normalizes the formula into a canonical form chosen by a profile. Normalization is gated on a complete tree — a tree with errors cannot be normalized. See [Transforms](https://texform.dev/docs/transforms).
* **`to_syntax()`** converts the tree back to a `SyntaxNode` for structured-data output. This is the only exit that produces serde-friendly structured data; there is no method named `serialize` on `Document`.

- [What TeXForm is](https://texform.dev/docs/what-is-texform) — The product boundary: what TeXForm does and what it leaves to other tools.

- [Parsing](https://texform.dev/docs/parsing) — Why parsing is knowledge-driven and what a parse result can contain.

- [Transforms](https://texform.dev/docs/transforms) — Profiles and the normalization pipeline behind the transform engine.

- [Serialization](https://texform.dev/docs/serialization) — The output channels, text idempotency, and output style.