# Transforms

How TeXForm normalizes a formula: pick a profile, run the phase loop to a fixed point, read the report, and switch phases off per run.

https://texform.dev/docs/transforms

Normalizing a LaTeX formula has no single correct answer. The form that is ideal for comparing two formulas for equivalence is wrong for an author who wants their shorthand preserved, and the form that strips layout hints for a training corpus would damage author-facing output. So TeXForm never imposes one canonical form. You pick a **profile** that names your downstream use, and a curated rule pipeline produces the canonical form for that use.

The transform engine is not a find-and-replace pass. It is a profile selection on top of a multi-phase pipeline driven by a declarative rule set, where every rule declares what it normalizes and what it guarantees. This page follows the four things you actually do with it: pick a profile, understand the pipeline it drives, request its diagnostic report, and switch individual phases off for one run.

## Pick a profile [#pick-a-profile]

A `TransformEngine` is built around one profile and normalizes formulas into that profile's canonical form.

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"a \over b"))
```

```text
\frac { a } { b }
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('a \\over b'));
```

```text
\frac { a } { b }
```

`normalize` returns the normalized string. The diagnostic report is a separate call, described in Reading the report below. Build an engine once per profile and reuse it across many formulas; the rule plan is compiled when the engine is constructed.

Each profile targets one downstream scenario:

| Profile     | Target scenario                                                                                       |
| ----------- | ----------------------------------------------------------------------------------------------------- |
| `Authoring` | Polished author-facing output; stylistic choices kept.                                                |
| `Faithful`  | Same rendered formula; convenience macros expanded into universal forms.                              |
| `Corpus`    | Complete canonical forms suitable as training labels; reading-identical layout variants may collapse. |
| `Equiv`     | Aggressive intermediates for equivalence comparison, deduplication, or fingerprints.                  |

The profiles form a ladder. Each one applies every rule the profile above it applies and then normalizes more aggressively, trading away progressively more of the source's surface form. `Authoring` keeps the author's intent; `Faithful` expands convenience macros while preserving rendering; `Corpus` additionally collapses reading-identical forms into complete training labels; `Equiv` adds rules that may discard visually salient notation choices.

The same input under all four profiles makes the ladder concrete (real engine output, with `physics` loaded so `\dv` is known):

| Input                                 | `Authoring`                             | `Faithful`                                      | `Corpus`                                               | `Equiv`                                                |
| ------------------------------------- | --------------------------------------- | ----------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------------ |
| `a \over b`                           | `\frac { a } { b }`                     | `\frac { a } { b }`                             | `\frac { a } { b }`                                    | `\frac { a } { b }`                                    |
| `\dv{f}{x}`                           | `\dv { f } { x }`                       | `\frac { \mathrm { d } f } { \mathrm { d } x }` | `\frac { \mathrm { d } f } { \mathrm { d } x }`        | `\frac { \mathrm { d } f } { \mathrm { d } x }`        |
| `\cos{A}`                             | `\cos { A }`                            | `\cos { A }`                                    | `\cos A`                                               | `\cos A`                                               |
| `\begin{pmatrix} a & b \end{pmatrix}` | `\begin {pmatrix} a & b \end {pmatrix}` | `\begin {pmatrix} a & b \end {pmatrix}`         | `\left ( \begin {matrix} a & b \end {matrix} \right )` | `\left ( \begin {matrix} a & b \end {matrix} \right )` |
| `\cfrac{a}{b}`                        | `\cfrac { a } { b }`                    | `\cfrac { a } { b }`                            | `\cfrac { a } { b }`                                   | `\frac { a } { b }`                                    |
| `\widehat{x}`                         | `\widehat { x }`                        | `\widehat { x }`                                | `\widehat { x }`                                       | `\hat { x }`                                           |

Each row isolates one rung of the ladder:

* `a \over b` modernizes legacy infix syntax into `\frac` under **every** profile. This is uncontroversial standardization that no scenario wants to keep.
* `\dv{f}{x}` is a `physics` convenience macro. `Authoring` keeps the shorthand; `Faithful` and stronger profiles expand it into the universal `\frac{\mathrm{d} f}{\mathrm{d} x}` form that renders without the `physics` package.
* `\cos{A}` keeps its redundant braces under `Authoring` and `Faithful`, which preserve spacing context, but drops them under `Corpus` and `Equiv`, which strip spacing hints a corpus should not learn.
* `\begin{pmatrix}` enters at `Corpus`, which rewrites the fenced matrix into explicit `\left(` / `\right)` delimiters around a bare `matrix`, so reading-identical specialized spellings collapse to one form.
* `\cfrac{a}{b}` and `\widehat{x}` enter at `Equiv`. Both rewrites lose something a reader can see — the centered continued-fraction layout, the accent stretched to its base's width — so `Corpus` keeps them as valid labels while `Equiv` folds them into `\frac` and `\hat`.

To choose: cleaning a training corpus selects `Corpus`; comparing formulas for equivalence selects `Equiv` (paired with a CAS for true mathematical equivalence, covered below); keeping the rendered formula unchanged while expanding macros selects `Faithful`; producing author-facing output selects `Authoring`. The decision is downstream-driven, not a quality dial — a "stronger" profile is not a "better" profile. `Equiv` output is the wrong training label for the image it came from, because comparison-only rules need not preserve notation; `Authoring` output is the wrong input for an equivalence check, because equivalent formulas can still differ as strings. Pick the profile whose target scenario matches what you will do with the output.

## The pipeline [#the-pipeline]

Behind a profile, normalization runs as an ordered sequence of phases. The profile selects which rules are active; the phase order itself is fixed.

Normalization repeats LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups until all enabled phases reach a common fixed point. A read-only eliminated-form check follows.

Each round follows LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups. Each phase reaches its own fixed point; any mutation invalidates the other phases. A phase that has already processed the current tree is skipped. Rounds continue until all enabled phases agree on a fixed point, with a bounded round limit that returns a convergence error rather than looping forever. The eliminated-form check runs once afterward: it reads the final tree and changes nothing.

The following sections describe each distinct phase with a verified before/after example.

### Rewrite [#rewrite]

Rewrite applies the profile's rule set in a fixed-point loop: it keeps applying rules until an iteration changes nothing or the iteration cap is reached. Each rule is declarative — it states the forms it watches for, the forms it eliminates, and the forms it produces — so the engine schedules rules rather than running a fixed script.

This phase is where legacy syntax is modernized and convenience macros are expanded. `a \over b` becomes `\frac { a } { b }`; `\dv{f}{x}` becomes `\frac { \mathrm { d } f } { \mathrm { d } x }` under `Faithful` and stronger. Which rules are active is decided by the profile's rule levels, described under [What the engine guarantees](#what-the-engine-guarantees).

The fixed-point loop matters when one rule's output is another rule's input. Expanding `\dv{f}{x}` produces a `\frac` whose arguments contain a `\mathrm` wrapper; a later iteration can then normalize that wrapper further. The engine keeps iterating until a full pass changes nothing, which is why the report for even a single-rule input often shows more than one iteration — the final pass is the one that confirms the tree has stabilized. The loop is bounded by a configurable cap so a misbehaving rule set cannot spin forever.

### LowerAttributes [#lowerattributes]

LowerAttributes canonicalizes font and style markup so that the two LaTeX ways of writing the same styling converge to one form. The declarative form `{\bf x}` and the prefix form `\mathbf{x}` both become `\mathbf { x }`:

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"{\bf x}"))
print(engine.normalize(r"\mathbf{x}"))
```

```text
\mathbf { x }
\mathbf { x }
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('{\\bf x}'));
console.log(engine.normalize('\\mathbf{x}'));
```

```text
\mathbf { x }
\mathbf { x }
```

The phase tracks declarative scope across container boundaries, so a declarative command's effect is bounded correctly instead of leaking into following siblings.

### FinalizeAst [#finalizeast]

FinalizeAst is the profile-neutral cleanup pass: it canonicalizes the tree's shape without consulting any rule metadata. It does two kinds of work. It merges adjacent `Prime` nodes, so a prime written as `\prime` marks and a prime written as quote shorthand reach the same final shape. And it normalizes text sequences — merging adjacent text-mode siblings, collapsing runs of ordinary whitespace to a single space without trimming the edges, and cleaning out empty text.

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"f^{\prime\prime}"))
print(engine.normalize(r"f''"))
print(engine.normalize(r"\text{a{}b}"))
```

```text
f''
f''
\text {ab}
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('f^{\\prime\\prime}'));
console.log(engine.normalize("f''"));
console.log(engine.normalize('\\text{a{}b}'));
```

```text
f''
f''
\text {ab}
```

The pass is idempotent: revisiting an already-canonical tree does not count additional mutations.

### FlattenGroups [#flattengroups]

FlattenGroups removes structurally redundant braces — empty groups, single-child groups in argument slots, and nested groups that add no structure — once the earlier phases have stabilized. Redundant nesting like `{{x}}` collapses to `x` under every profile:

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"{{x}}"))
```

```text
x
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('{{x}}'));
```

```text
x
```

Flattening is gated so it never changes meaning by accident. The public FlattenGroups switches are `enabled` and `preserve_rendered_spacing`. Structural (semantic) guards stay on whenever the phase runs: they protect script binding, environment cell boundaries, declarative scope, infix scope, the brace groups after an unknown command, and a group whose opening delimiter the command before it would otherwise read as an argument. With `physics` loaded, for example, `\sin{[2]}x` keeps its braces, because `\sin[2]x` would read `2` as the exponent. `preserve_rendered_spacing` turns the spacing guards on or off together: `Authoring` and `Faithful` keep it true; `Corpus` and `Equiv` set it false. That is exactly why `\cos{A}` keeps its braces under `Authoring` but loses them under `Corpus`.

## What the engine guarantees [#what-the-engine-guarantees]

Two mechanisms back the claim that normalization is correct rather than best-effort.

**Normalization is gated on a complete tree.** A tree with `Error` nodes cannot be normalized — normalizing a formula with holes is meaningless — so the engine refuses it. This is why the transform path begins by ensuring the parse is clean:

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
try:
    engine.normalize(r"x^")
except texform.ParseError as exc:
    print("rejected:", exc)
```

```text
rejected: parse produced an incomplete document
```

**JavaScript**

```js
import { TransformEngine, TexformParseError } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
try {
  engine.normalize('x^');
} catch (exc) {
  console.log('rejected:', exc instanceof TexformParseError);
}
```

```text
rejected: true
```

The same gate applies to `transform`, which normalizes a live `Document` in place: it accepts only complete documents that came from its own engine's parser.

**The eliminated-form contract is enforced, not assumed.** Each rewrite rule declares the forms it eliminates — forms that must not appear anywhere in the output. After every mutating phase has run, the engine checks that no eliminated form survives. A remaining eliminated form is a hard transform error surfaced as a contract violation, not a silent warning. The same contract is re-checked continuously against real-world datasets through corpus regression, so a rule that fails to eliminate what it promised is caught against actual formulas, not only unit tests.

A rule's **rule level** is what decides whether a profile runs it. `RuleLevel` is ordered `Authoring < Faithful < Corpus < Equiv`, and a rule's level is the first profile that accepts the rule's output as a suitable product: `Authoring` rules run under every profile; `Faithful` rules run under Faithful and up; `Corpus` rules under Corpus and up; `Equiv` rules only under Equiv. Each profile turns on its own level plus every level below it, so moving up the ladder only adds rules.

A rule's **fidelity** is a separate worst-case equivalence guarantee, ordered `Math < Reading < Render`. Level answers "which use case first accepts this output"; `RuleFidelity` answers "which equivalence relation holds over the declared input domain". Authoring, Faithful, and Corpus require at least `Reading`, while Equiv permits `Math`. A stronger fidelity does not automatically place a rule in an earlier level. The rule that drops fixed delimiter sizing, turning `\big( x \big)` into `( x )`, declares `Reading` fidelity — notation identity, reading order, and structural roles all survive — and is still an `Equiv` rule, because a training label should keep the size the author chose. You can browse every rule with its level and fidelity in the [rule reference](https://texform.dev/rules).

## Reading the report [#reading-the-report]

Plain `normalize` returns the canonical string and does not build a report. Plain `transform` updates a document in place and returns nothing (`None` in Python, `void` in JavaScript). Ask for the diagnostic report with the same source and configuration on a separate method: `normalize_with_report` / `transform_with_report` in Python, and `normalizeWithReport` / `transformWithReport` in JavaScript. `normalize_with_report` and `normalizeWithReport` return a `NormalizeReportResult` with `normalized` and `report`. The transform report methods return the report alone. Output text and errors follow the plain transform contract. Report fields, phase grouping, and counters are diagnostic and are not a stable compatibility promise.

The report has one section per phase: `lower_attributes`, `rewrite` (`iterations` and `rules`), `finalize_ast` (`prime_run_merges` and `text_normalizations`), and `flatten_groups` (`actions` and `guard_hits`). `guard_hits` has one counter per preserve guard and structural protection: `unknown_command_arguments`, `argument_capture`, `declarative_scope`, `script_base`, `env_body`, `infix_scope`, `command_contact`, `command_contact_via_scripted_base`, `empty_group`, `lone_atom_spacing_char`, `leading_atom_spacing_char`, and `delimited_pair`. `command_contact_via_scripted_base` is also included in `command_contact`, so the two are not independent totals; do not add them.

This input exercises four phases at once — a declarative font command, a legacy infix, two `\prime` commands, and an empty text group:

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")
result = engine.normalize_with_report(r"{\bf x} + {a \over b} + f^{\prime\prime} + \text{a{}b}")
report = result["report"]

print(result["normalized"])
print("iterations:", report["rewrite"]["iterations"])
print("rules:", [(r["key"], r["applied_count"]) for r in report["rewrite"]["rules"]])
print("finalize_ast:", report["finalize_ast"])
print("flatten_groups:", report["flatten_groups"]["actions"])
print("lower_attributes:", report["lower_attributes"]["attributes"])
```

```text
\mathbf { x } + \frac { a } { b } + f'' + \text {ab}
iterations: 4
rules: [('base/over-to-frac', 1), ('base/prime-to-prime-node', 2)]
finalize_ast: {'prime_run_merges': 1, 'text_normalizations': 1}
flatten_groups: {'removed_empty': 1, 'replaced_single_child': 0, 'inlined_multi_child': 0, 'unwrapped_slot': 2}
lower_attributes: [{'attr': 'math_font', 'value': 'bold', 'consumed': {'declaratives': 1, 'prefixes': 3}, 'redundant': {'declaratives': 0, 'prefixes': 0}, 'emitted': {'declaratives': 0, 'prefixes': 4}}]
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
const result = engine.normalizeWithReport('{\\bf x} + {a \\over b} + f^{\\prime\\prime} + \\text{a{}b}');
const report = result.report;

console.log(result.normalized);
console.log('iterations:', report.rewrite.iterations);
console.log('rules:', report.rewrite.rules.map((r) => [r.key, r.appliedCount]));
console.log('finalizeAst:', report.finalizeAst);
console.log('flattenGroups:', report.flattenGroups.actions);
console.log('lowerAttributes:', report.lowerAttributes.attributes);
```

Reading it phase by phase:

* **`rewrite.iterations`** counts Rewrite iterations across the phase rounds: 4 on this input. The count includes unchanged convergence checks after other phases mutate the tree.
* **`rewrite.rules`** lists each rule that recorded activity, with how many times it applied. Here `base/over-to-frac` fired once, turning `a \over b` into `\frac`, and `base/prime-to-prime-node` fired twice, turning each `\prime` command into a `Prime` node.
* **`finalize_ast.prime_run_merges`** fired once, fusing the two `Prime` nodes Rewrite had just produced into the `f''` shorthand. **`finalize_ast.text_normalizations`** fired once, removing the empty group inside `\text{a{}b}`.
* **`flatten_groups.actions`** records one `removed_empty` and two `unwrapped_slot` removals: the empty `{}` is gone, and the braces wrapping `a \over b` and the `\prime` script were redundant single-child groups in argument and script slots. **`flatten_groups.guard_hits`** counts the first preserve branch reached for a group; every counter is zero on this input.
* **`lower_attributes.attributes`** shows the `math_font: bold` unit. The counters accumulate across all phase rounds: the first pass consumes the `{\bf x}` declarative and emits the `\mathbf{x}` prefix form; later visits re-canonicalize that prefix as other phases change the tree. Here `consumed` totals one declarative and three prefixes, while `emitted` totals four prefixes.

JavaScript reports the same data with camelCase field names — `finalizeAst`, `primeRunMerges`, `textNormalizations`, `guardHits`, `removedEmpty` — and each rule entry uses `key`, `appliedCount`, `skippedCount`. A disabled phase stays in the report as zeros or an empty list. These counters are not a single "number of edits": do not add them together and read the sum as output changes.

## Switch a phase off with an overlay [#switch-a-phase-off-with-an-overlay]

A profile sets sensible defaults, but you can override individual phases for one run by passing an **overlay**: a partial, per-phase set of switches merged onto the profile's own config. Disabling Rewrite leaves legacy syntax untouched; disabling FlattenGroups keeps redundant braces.

**Python**

```python
import texform

engine = texform.TransformEngine(profile="corpus")

# Overlays are keyword arguments holding per-phase dicts.
print(engine.normalize(r"a \over b", rewrite={"enabled": False}))
print(engine.normalize(r"\cos{A}", flatten_groups={"enabled": False}))

# Or start from the profile's complete config and change a field.
config = texform.TransformConfig.corpus()
config.rewrite.max_iterations = 50
print(engine.normalize(r"a \over b", config=config))
```

```text
a \over b
\cos { A }
\frac { a } { b }
```

**JavaScript**

```js
import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });

// The overlay is a nested, camelCase object; one key per phase.
console.log(engine.normalize('a \\over b', { rewrite: { enabled: false } }));
console.log(engine.normalize('\\cos{A}', { flattenGroups: { enabled: false } }));
```

```text
a \over b
\cos { A }
```

The two languages differ in how they express a *complete* config, and the difference is worth knowing before you copy code between them. In Python, `config=` takes a `TransformConfig` instance whose four phase fields are all required, so you build one with `TransformConfig.authoring()`, `faithful()`, `corpus()`, or `equiv()` and then adjust fields; passing a plain dict as `config` raises a configuration error. In JavaScript there is no complete config class at all — every per-run object is an overlay, and `defaultTransformConfig()` is how you read the effective values. In both languages the overlay is nested one level per phase: `lower_attributes` / `lowerAttributes`, `rewrite`, `finalize_ast` / `finalizeAst`, `flatten_groups` / `flattenGroups`. An unknown key, or a Python-style key handed to the JavaScript binding, raises a configuration error rather than being silently ignored.

An overlay changes *which phases run*, never which rules the profile selected. If you want the pipeline intact but one specific rewrite rule out of it, disable that rule when you construct the engine instead — `disable_rules=["physics/dv-to-frac-d"]` in Python, `disableRules` in JavaScript. The full set of switches is documented on the [transform engine API page](https://texform.dev/api/python/transform-engine).

## What the engine does not do [#what-the-engine-does-not-do]

The transform engine normalizes the surface form of a formula. It does not perform mathematical reasoning: it will not prove that `a + b` equals `b + a`, evaluate an integral, or simplify an algebraic expression. The `Equiv` profile narrows the gap by canonicalizing equivalent spellings to the same string, so a string comparison catches many cases for free — but true mathematical equivalence needs a computer algebra system. Treat `Equiv` as a cheap first pass that resolves spelling differences, and reserve a CAS for the cases that survive it.

The engine also does not render: it produces LaTeX, not pixels. Fidelity levels are metadata contracts the engine declares, not comparisons it runs; verifying that two formulas render identically is the job of a separate renderer-validation step, not of normalization.

- [Glossary](https://texform.dev/docs/glossary) — Profile, rule level, fidelity, overlay, phase — the vocabulary on this page, defined.

- [Serialization](https://texform.dev/docs/serialization) — The output channels and text idempotency behind normalization.

- [Normalize a corpus](https://texform.dev/docs/normalize-a-corpus) — Run the engine over a dataset, with error handling and statistics.

- [Compare formulas](https://texform.dev/docs/compare-formulas) — Use the equiv profile with a CAS for equivalence checking.