Transforms

Normalizing a LaTeX formula has no single correct answer. The form that is ideal for comparing two formulas for equivalence is wrong for an author who wants their shorthand preserved, and the form that strips layout hints for a training corpus would damage author-facing output. So TeXForm never imposes one canonical form. You pick a profile that names your downstream use, and a curated rule pipeline produces the canonical form for that use.

The transform engine is not a find-and-replace pass. It is a profile selection on top of a multi-phase pipeline driven by a declarative rule set, where every rule declares what it normalizes and what it guarantees. This page follows the four things you actually do with it: pick a profile, understand the pipeline it drives, request its diagnostic report, and switch individual phases off for one run.

Pick a profile#

A TransformEngine is built around one profile and normalizes formulas into that profile's canonical form.

import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"a \over b"))

\frac { a } { b }

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('a \\over b'));

\frac { a } { b }

normalize returns the normalized string. The diagnostic report is a separate call, described in Reading the report below. Build an engine once per profile and reuse it across many formulas; the rule plan is compiled when the engine is constructed.

Each profile targets one downstream scenario:

The profiles form a ladder. Each one applies every rule the profile above it applies and then normalizes more aggressively, trading away progressively more of the source's surface form. Authoring keeps the author's intent; Faithful expands convenience macros while preserving rendering; Corpus additionally collapses reading-identical forms into complete training labels; Equiv adds rules that may discard visually salient notation choices.

The same input under all four profiles makes the ladder concrete (real engine output, with physics loaded so \dv is known):

Each row isolates one rung of the ladder:

• a \over b modernizes legacy infix syntax into \frac under every profile. This is uncontroversial standardization that no scenario wants to keep.
• \dv{f}{x} is a physics convenience macro. Authoring keeps the shorthand; Faithful and stronger profiles expand it into the universal \frac{\mathrm{d} f}{\mathrm{d} x} form that renders without the physics package.
• \cos{A} keeps its redundant braces under Authoring and Faithful, which preserve spacing context, but drops them under Corpus and Equiv, which strip spacing hints a corpus should not learn.
• \begin{pmatrix} enters at Corpus, which rewrites the fenced matrix into explicit \left( / \right) delimiters around a bare matrix, so reading-identical specialized spellings collapse to one form.
• \cfrac{a}{b} and \widehat{x} enter at Equiv. Both rewrites lose something a reader can see — the centered continued-fraction layout, the accent stretched to its base's width — so Corpus keeps them as valid labels while Equiv folds them into \frac and \hat.

To choose: cleaning a training corpus selects Corpus; comparing formulas for equivalence selects Equiv (paired with a CAS for true mathematical equivalence, covered below); keeping the rendered formula unchanged while expanding macros selects Faithful; producing author-facing output selects Authoring. The decision is downstream-driven, not a quality dial — a "stronger" profile is not a "better" profile. Equiv output is the wrong training label for the image it came from, because comparison-only rules need not preserve notation; Authoring output is the wrong input for an equivalence check, because equivalent formulas can still differ as strings. Pick the profile whose target scenario matches what you will do with the output.

The pipeline#

Behind a profile, normalization runs as an ordered sequence of phases. The profile selects which rules are active; the phase order itself is fixed.

Each round follows LowerAttributes → Rewrite → LowerAttributes → FinalizeAst → FlattenGroups. Each phase reaches its own fixed point; any mutation invalidates the other phases. A phase that has already processed the current tree is skipped. Rounds continue until all enabled phases agree on a fixed point, with a bounded round limit that returns a convergence error rather than looping forever. The eliminated-form check runs once afterward: it reads the final tree and changes nothing.

The following sections describe each distinct phase with a verified before/after example.

Rewrite#

Rewrite applies the profile's rule set in a fixed-point loop: it keeps applying rules until an iteration changes nothing or the iteration cap is reached. Each rule is declarative — it states the forms it watches for, the forms it eliminates, and the forms it produces — so the engine schedules rules rather than running a fixed script.

This phase is where legacy syntax is modernized and convenience macros are expanded. a \over b becomes \frac { a } { b }; \dv{f}{x} becomes \frac { \mathrm { d } f } { \mathrm { d } x } under Faithful and stronger. Which rules are active is decided by the profile's rule levels, described under What the engine guarantees.

The fixed-point loop matters when one rule's output is another rule's input. Expanding \dv{f}{x} produces a \frac whose arguments contain a \mathrm wrapper; a later iteration can then normalize that wrapper further. The engine keeps iterating until a full pass changes nothing, which is why the report for even a single-rule input often shows more than one iteration — the final pass is the one that confirms the tree has stabilized. The loop is bounded by a configurable cap so a misbehaving rule set cannot spin forever.

LowerAttributes#

LowerAttributes canonicalizes font and style markup so that the two LaTeX ways of writing the same styling converge to one form. The declarative form {\bf x} and the prefix form \mathbf{x} both become \mathbf { x }:

import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"{\bf x}"))
print(engine.normalize(r"\mathbf{x}"))

\mathbf { x }
\mathbf { x }

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('{\\bf x}'));
console.log(engine.normalize('\\mathbf{x}'));

\mathbf { x }
\mathbf { x }

The phase tracks declarative scope across container boundaries, so a declarative command's effect is bounded correctly instead of leaking into following siblings.

FinalizeAst#

FinalizeAst is the profile-neutral cleanup pass: it canonicalizes the tree's shape without consulting any rule metadata. It does two kinds of work. It merges adjacent Prime nodes, so a prime written as \prime marks and a prime written as quote shorthand reach the same final shape. And it normalizes text sequences — merging adjacent text-mode siblings, collapsing runs of ordinary whitespace to a single space without trimming the edges, and cleaning out empty text.

import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"f^{\prime\prime}"))
print(engine.normalize(r"f''"))
print(engine.normalize(r"\text{a{}b}"))

f''
f''
\text {ab}

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('f^{\\prime\\prime}'));
console.log(engine.normalize("f''"));
console.log(engine.normalize('\\text{a{}b}'));

f''
f''
\text {ab}

The pass is idempotent: revisiting an already-canonical tree does not count additional mutations.

FlattenGroups#

FlattenGroups removes structurally redundant braces — empty groups, single-child groups in argument slots, and nested groups that add no structure — once the earlier phases have stabilized. Redundant nesting like {{x}} collapses to x under every profile:

import texform

engine = texform.TransformEngine(profile="corpus")
print(engine.normalize(r"{{x}}"))

x

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
console.log(engine.normalize('{{x}}'));

x

Flattening is gated so it never changes meaning by accident. The public FlattenGroups switches are enabled and preserve_rendered_spacing. Structural (semantic) guards stay on whenever the phase runs: they protect script binding, environment cell boundaries, declarative scope, infix scope, the brace groups after an unknown command, and a group whose opening delimiter the command before it would otherwise read as an argument. With physics loaded, for example, \sin{[2]}x keeps its braces, because \sin[2]x would read 2 as the exponent. preserve_rendered_spacing turns the spacing guards on or off together: Authoring and Faithful keep it true; Corpus and Equiv set it false. That is exactly why \cos{A} keeps its braces under Authoring but loses them under Corpus.

What the engine guarantees#

Two mechanisms back the claim that normalization is correct rather than best-effort.

Normalization is gated on a complete tree. A tree with Error nodes cannot be normalized — normalizing a formula with holes is meaningless — so the engine refuses it. This is why the transform path begins by ensuring the parse is clean:

import texform

engine = texform.TransformEngine(profile="corpus")
try:
    engine.normalize(r"x^")
except texform.ParseError as exc:
    print("rejected:", exc)

rejected: parse produced an incomplete document

import { TransformEngine, TexformParseError } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
try {
  engine.normalize('x^');
} catch (exc) {
  console.log('rejected:', exc instanceof TexformParseError);
}

rejected: true

The same gate applies to transform, which normalizes a live Document in place: it accepts only complete documents that came from its own engine's parser.

The eliminated-form contract is enforced, not assumed. Each rewrite rule declares the forms it eliminates — forms that must not appear anywhere in the output. After every mutating phase has run, the engine checks that no eliminated form survives. A remaining eliminated form is a hard transform error surfaced as a contract violation, not a silent warning. The same contract is re-checked continuously against real-world datasets through corpus regression, so a rule that fails to eliminate what it promised is caught against actual formulas, not only unit tests.

A rule's rule level is what decides whether a profile runs it. RuleLevel is ordered Authoring < Faithful < Corpus < Equiv, and a rule's level is the first profile that accepts the rule's output as a suitable product: Authoring rules run under every profile; Faithful rules run under Faithful and up; Corpus rules under Corpus and up; Equiv rules only under Equiv. Each profile turns on its own level plus every level below it, so moving up the ladder only adds rules.

A rule's fidelity is a separate worst-case equivalence guarantee, ordered Math < Reading < Render. Level answers "which use case first accepts this output"; RuleFidelity answers "which equivalence relation holds over the declared input domain". Authoring, Faithful, and Corpus require at least Reading, while Equiv permits Math. A stronger fidelity does not automatically place a rule in an earlier level. The rule that drops fixed delimiter sizing, turning \big( x \big) into ( x ), declares Reading fidelity — notation identity, reading order, and structural roles all survive — and is still an Equiv rule, because a training label should keep the size the author chose. You can browse every rule with its level and fidelity in the rule reference.

Reading the report#

Plain normalize returns the canonical string and does not build a report. Plain transform updates a document in place and returns nothing (None in Python, void in JavaScript). Ask for the diagnostic report with the same source and configuration on a separate method: normalize_with_report / transform_with_report in Python, and normalizeWithReport / transformWithReport in JavaScript. normalize_with_report and normalizeWithReport return a NormalizeReportResult with normalized and report. The transform report methods return the report alone. Output text and errors follow the plain transform contract. Report fields, phase grouping, and counters are diagnostic and are not a stable compatibility promise.

The report has one section per phase: lower_attributes, rewrite (iterations and rules), finalize_ast (prime_run_merges and text_normalizations), and flatten_groups (actions and guard_hits). guard_hits has one counter per preserve guard and structural protection: unknown_command_arguments, argument_capture, declarative_scope, script_base, env_body, infix_scope, command_contact, command_contact_via_scripted_base, empty_group, lone_atom_spacing_char, leading_atom_spacing_char, and delimited_pair. command_contact_via_scripted_base is also included in command_contact, so the two are not independent totals; do not add them.

This input exercises four phases at once — a declarative font command, a legacy infix, two \prime commands, and an empty text group:

import texform

engine = texform.TransformEngine(profile="corpus")
result = engine.normalize_with_report(r"{\bf x} + {a \over b} + f^{\prime\prime} + \text{a{}b}")
report = result["report"]

print(result["normalized"])
print("iterations:", report["rewrite"]["iterations"])
print("rules:", [(r["key"], r["applied_count"]) for r in report["rewrite"]["rules"]])
print("finalize_ast:", report["finalize_ast"])
print("flatten_groups:", report["flatten_groups"]["actions"])
print("lower_attributes:", report["lower_attributes"]["attributes"])

\mathbf { x } + \frac { a } { b } + f'' + \text {ab}
iterations: 4
rules: [('base/over-to-frac', 1), ('base/prime-to-prime-node', 2)]
finalize_ast: {'prime_run_merges': 1, 'text_normalizations': 1}
flatten_groups: {'removed_empty': 1, 'replaced_single_child': 0, 'inlined_multi_child': 0, 'unwrapped_slot': 2}
lower_attributes: [{'attr': 'math_font', 'value': 'bold', 'consumed': {'declaratives': 1, 'prefixes': 3}, 'redundant': {'declaratives': 0, 'prefixes': 0}, 'emitted': {'declaratives': 0, 'prefixes': 4}}]

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });
const result = engine.normalizeWithReport('{\\bf x} + {a \\over b} + f^{\\prime\\prime} + \\text{a{}b}');
const report = result.report;

console.log(result.normalized);
console.log('iterations:', report.rewrite.iterations);
console.log('rules:', report.rewrite.rules.map((r) => [r.key, r.appliedCount]));
console.log('finalizeAst:', report.finalizeAst);
console.log('flattenGroups:', report.flattenGroups.actions);
console.log('lowerAttributes:', report.lowerAttributes.attributes);

Reading it phase by phase:

• rewrite.iterations counts Rewrite iterations across the phase rounds: 4 on this input. The count includes unchanged convergence checks after other phases mutate the tree.
• rewrite.rules lists each rule that recorded activity, with how many times it applied. Here base/over-to-frac fired once, turning a \over b into \frac, and base/prime-to-prime-node fired twice, turning each \prime command into a Prime node.
• finalize_ast.prime_run_merges fired once, fusing the two Prime nodes Rewrite had just produced into the f'' shorthand. finalize_ast.text_normalizations fired once, removing the empty group inside \text{a{}b}.
• flatten_groups.actions records one removed_empty and two unwrapped_slot removals: the empty {} is gone, and the braces wrapping a \over b and the \prime script were redundant single-child groups in argument and script slots. flatten_groups.guard_hits counts the first preserve branch reached for a group; every counter is zero on this input.
• lower_attributes.attributes shows the math_font: bold unit. The counters accumulate across all phase rounds: the first pass consumes the {\bf x} declarative and emits the \mathbf{x} prefix form; later visits re-canonicalize that prefix as other phases change the tree. Here consumed totals one declarative and three prefixes, while emitted totals four prefixes.

JavaScript reports the same data with camelCase field names — finalizeAst, primeRunMerges, textNormalizations, guardHits, removedEmpty — and each rule entry uses key, appliedCount, skippedCount. A disabled phase stays in the report as zeros or an empty list. These counters are not a single "number of edits": do not add them together and read the sum as output changes.

Switch a phase off with an overlay#

A profile sets sensible defaults, but you can override individual phases for one run by passing an overlay: a partial, per-phase set of switches merged onto the profile's own config. Disabling Rewrite leaves legacy syntax untouched; disabling FlattenGroups keeps redundant braces.

import texform

engine = texform.TransformEngine(profile="corpus")

# Overlays are keyword arguments holding per-phase dicts.
print(engine.normalize(r"a \over b", rewrite={"enabled": False}))
print(engine.normalize(r"\cos{A}", flatten_groups={"enabled": False}))

# Or start from the profile's complete config and change a field.
config = texform.TransformConfig.corpus()
config.rewrite.max_iterations = 50
print(engine.normalize(r"a \over b", config=config))

a \over b
\cos { A }
\frac { a } { b }

import { TransformEngine } from 'texform';

const engine = new TransformEngine({ profile: 'corpus' });

// The overlay is a nested, camelCase object; one key per phase.
console.log(engine.normalize('a \\over b', { rewrite: { enabled: false } }));
console.log(engine.normalize('\\cos{A}', { flattenGroups: { enabled: false } }));

a \over b
\cos { A }

The two languages differ in how they express a complete config, and the difference is worth knowing before you copy code between them. In Python, config= takes a TransformConfig instance whose four phase fields are all required, so you build one with TransformConfig.authoring(), faithful(), corpus(), or equiv() and then adjust fields; passing a plain dict as config raises a configuration error. In JavaScript there is no complete config class at all — every per-run object is an overlay, and defaultTransformConfig() is how you read the effective values. In both languages the overlay is nested one level per phase: lower_attributes / lowerAttributes, rewrite, finalize_ast / finalizeAst, flatten_groups / flattenGroups. An unknown key, or a Python-style key handed to the JavaScript binding, raises a configuration error rather than being silently ignored.

An overlay changes which phases run, never which rules the profile selected. If you want the pipeline intact but one specific rewrite rule out of it, disable that rule when you construct the engine instead — disable_rules=["physics/dv-to-frac-d"] in Python, disableRules in JavaScript. The full set of switches is documented on the transform engine API page.

What the engine does not do#

The transform engine normalizes the surface form of a formula. It does not perform mathematical reasoning: it will not prove that a + b equals b + a, evaluate an integral, or simplify an algebraic expression. The Equiv profile narrows the gap by canonicalizing equivalent spellings to the same string, so a string comparison catches many cases for free — but true mathematical equivalence needs a computer algebra system. Treat Equiv as a cheap first pass that resolves spelling differences, and reserve a CAS for the cases that survive it.

The engine also does not render: it produces LaTeX, not pixels. Fidelity levels are metadata contracts the engine declares, not comparisons it runs; verifying that two formulas render identically is the job of a separate renderer-validation step, not of normalization.